You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2013/01/07 19:00:11 UTC
A couple of unit tests may be using a data dir of /tmp/hbase-${user}
There was a failure of TestLocalHBaseCluster in
https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able to
reproduce locally, but what I think this means is a couple of tests are
using a data dir of /tmp/hbase-${user}.
I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed it
back to 2 after review advice from Ted. TestLocalHBaseCluster then started
failing because some HFiles with a version of 2.3 remained in the
filesystem under /tmp/hbase-${user} from earlier tests and could not be
read in by current tests. The precommit builds don't save the test logs so
I can't confirm if what is going on up on Jenkins is the same thing, but I
suspect so.
I'm not sure if JUnit or Surefire can support a global post test action,
but I'll look into it. I want to test for the existence
of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
it as an offender.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: A couple of unit tests may be using a data dir of /tmp/hbase-${user}
Posted by Enis Söztutar <en...@gmail.com>.
For CP's, we have a candidate patch at:
http://issues.apache.org/jira/browse/HBASE-6824. We can use this patch to
migrate the remaining usages from the main code side, and also set this in
HBTU.
Enis
On Mon, Jan 7, 2013 at 11:42 AM, Nicolas Liochon <nk...@gmail.com> wrote:
> There are tests writing stuff like
> /tmp/..-577547068.TestCP3.1357574431574.jar.crc or
> /tmp/..-1797055647.TestCPInvalid.1357574424182.jar.crc as well.
>
> Note the '.' to make them invisible :-)
>
>
> On Mon, Jan 7, 2013 at 8:38 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > should fix the tests themselves to not
> > use the /tmp directory. I guess we can use the ResourceChecker to detect
> > and fail those tests.
> >
>
Re: A couple of unit tests may be using a data dir of /tmp/hbase-${user}
Posted by Nicolas Liochon <nk...@gmail.com>.
There are tests writing stuff like
/tmp/..-577547068.TestCP3.1357574431574.jar.crc or
/tmp/..-1797055647.TestCPInvalid.1357574424182.jar.crc as well.
Note the '.' to make them invisible :-)
On Mon, Jan 7, 2013 at 8:38 PM, Enis Söztutar <en...@gmail.com> wrote:
> should fix the tests themselves to not
> use the /tmp directory. I guess we can use the ResourceChecker to detect
> and fail those tests.
>
Re: A couple of unit tests may be using a data dir of /tmp/hbase-${user}
Posted by Enis Söztutar <en...@gmail.com>.
Andrew, could you please review HBASE-6824 then. Let's get the ball
rolling.
Enis
On Mon, Jan 7, 2013 at 12:26 PM, Andrew Purtell <ap...@apache.org> wrote:
> Sounds perfect to me.
>
> On Mon, Jan 7, 2013 at 11:38 AM, Enis Söztutar <en...@gmail.com> wrote:
>
> > I guess we can use the ResourceChecker to detect and fail those tests.
> >
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: A couple of unit tests may be using a data dir of /tmp/hbase-${user}
Posted by Andrew Purtell <ap...@apache.org>.
Sounds perfect to me.
On Mon, Jan 7, 2013 at 11:38 AM, Enis Söztutar <en...@gmail.com> wrote:
> I guess we can use the ResourceChecker to detect and fail those tests.
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: A couple of unit tests may be using a data dir of /tmp/hbase-${user}
Posted by Enis Söztutar <en...@gmail.com>.
I've also noticed this before, but though we fixed all instances in 96.
Instead of jenkins cleaning up, we should fix the tests themselves to not
use the /tmp directory. I guess we can use the ResourceChecker to detect
and fail those tests.
Enis
On Mon, Jan 7, 2013 at 11:32 AM, Stack <st...@duboce.net> wrote:
> On Mon, Jan 7, 2013 at 10:00 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > There was a failure of TestLocalHBaseCluster in
> > https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able
> > to
> > reproduce locally, but what I think this means is a couple of tests are
> > using a data dir of /tmp/hbase-${user}.
> >
> > I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed
> it
> > back to 2 after review advice from Ted. TestLocalHBaseCluster then
> started
> > failing because some HFiles with a version of 2.3 remained in the
> > filesystem under /tmp/hbase-${user} from earlier tests and could not be
> > read in by current tests. The precommit builds don't save the test logs
> so
> > I can't confirm if what is going on up on Jenkins is the same thing, but
> I
> > suspect so.
> >
> > I'm not sure if JUnit or Surefire can support a global post test action,
> > but I'll look into it. I want to test for the existence
> > of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
> > it as an offender.
> >
>
>
> See
> ./hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
>
> Is it hdfs that is putting the files in /tmp?
>
> St.Ack
>
Re: A couple of unit tests may be using a data dir of /tmp/hbase-${user}
Posted by Andrew Purtell <ap...@apache.org>.
No, I'm seeing hbase-${user}
On Mon, Jan 7, 2013 at 11:32 AM, Stack <st...@duboce.net> wrote:
> On Mon, Jan 7, 2013 at 10:00 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > There was a failure of TestLocalHBaseCluster in
> > https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able
> > to
> > reproduce locally, but what I think this means is a couple of tests are
> > using a data dir of /tmp/hbase-${user}.
> >
> > I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed
> it
> > back to 2 after review advice from Ted. TestLocalHBaseCluster then
> started
> > failing because some HFiles with a version of 2.3 remained in the
> > filesystem under /tmp/hbase-${user} from earlier tests and could not be
> > read in by current tests. The precommit builds don't save the test logs
> so
> > I can't confirm if what is going on up on Jenkins is the same thing, but
> I
> > suspect so.
> >
> > I'm not sure if JUnit or Surefire can support a global post test action,
> > but I'll look into it. I want to test for the existence
> > of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
> > it as an offender.
> >
>
>
> See
> ./hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
>
> Is it hdfs that is putting the files in /tmp?
>
> St.Ack
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: A couple of unit tests may be using a data dir of /tmp/hbase-${user}
Posted by Stack <st...@duboce.net>.
On Mon, Jan 7, 2013 at 10:00 AM, Andrew Purtell <ap...@apache.org> wrote:
> There was a failure of TestLocalHBaseCluster in
> https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able
> to
> reproduce locally, but what I think this means is a couple of tests are
> using a data dir of /tmp/hbase-${user}.
>
> I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed it
> back to 2 after review advice from Ted. TestLocalHBaseCluster then started
> failing because some HFiles with a version of 2.3 remained in the
> filesystem under /tmp/hbase-${user} from earlier tests and could not be
> read in by current tests. The precommit builds don't save the test logs so
> I can't confirm if what is going on up on Jenkins is the same thing, but I
> suspect so.
>
> I'm not sure if JUnit or Surefire can support a global post test action,
> but I'll look into it. I want to test for the existence
> of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
> it as an offender.
>
See
./hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
Is it hdfs that is putting the files in /tmp?
St.Ack