You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2014/01/02 19:52:20 UTC
Re: Next week I would like to roll the first 0.98.0 RC
I have finished 50 runs of the unit test suite using JDK 6 and JDK 7 on
RHEL 6, Ubuntu 12, and Ubuntu 13. The combined results by JDK version are
as follows:
JDK 6 - Java(TM) SE Runtime Environment (build 1.6.0_43-b01) HotSpot(TM)
64-Bit Server VM (build 20.14-b01, mixed mode)
Count Test
5 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
3 org.apache.hadoop.hbase.TestIOFencing
2 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
1
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
1 org.apache.hadoop.hbase.TestNodeHealthCheckChore
1 org.apache.hadoop.hbase.procedure.TestZKProcedure
1 org.apache.hadoop.hbase.client.TestMultiParallel
JDK 7 - Java(TM) SE Runtime Environment (build 1.7.0_21-b11) HotSpot(TM)
64-Bit Server VM (build 23.21-b01, mixed mode)
Count Test
1 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
1 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
I opened HBASE-10267 for the TestNamespaceCommands failures and will work
on this since I bet it is related to recent changes I made in this area.
The TestSplitLogWorker failures were all under JDK 6 on Ubuntu 12. See
HBASE-10268.
I found five related issues for TestIOFencing failures: HBASE-8520,
HBASE-8646, HBASE-8889, HBASE-8907, and HBASE-9023. Therefore I am inclined
to disable this test on 0.98 branch for us to sort out on trunk later since
it's taking a while to get it right.
On Sun, Dec 29, 2013 at 3:59 PM, Andrew Purtell <ap...@apache.org> wrote:
> Next week I would like to roll the first 0.98.0 RC. Let's see.
>
> Some remaining work needs committing. Then, will work up a Google Docs
> spreadsheet for package checks, unit test suite runs, and cluster testing
> with hbase-it, LoadTestTool, and YCSB, and fill it in.
>
> The unit test suite is looking "ok". There are occasional failures on
> builds.apache.org, but these may have system load as the underlying
> cause. Unit test suite runs on more well endowed EC2 VMs are passing
> consistently. Let me add the caveat that I have so far not completed
> running through the full VM-OS matrix:
>
> VM:
>
>
> - Oracle JDK 6
> - Oracle JDK 7
> - OpenJDK 7
> - OpenJDK 8*
>
> OS (unit tests):
>
>
> - Amazon Linux
> - RHEL 6.4
> - Ubuntu Server 12.04.3 LTS
> - Ubuntu Server 13
>
> OS (cluster);
>
>
> - Amazon Linux
> - RHEL 6.5
>
> If I identify repeatable test failures under a particular combination,
> then I'll open a blocker JIRA for it. Regarding failures on
> builds.apache.org, if there are JIRAs filed for these we should certainly
> take a look but I suggest (and personally will be inclined to) weighting
> more highly evidence gathered from testing elsewhere.
>
> * - Java 8 tests are just for my own curiosity. Actually, you may be
> pleasantly surprised to know that all unit tests for HBase 0.98 and trunk
> were consistently passing on a Java 8 build a few weeks ago.
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Next week I would like to roll the first 0.98.0 RC
Posted by Ted Yu <yu...@gmail.com>.
I spent some time on making TestIOFencing pass last year.
I can re-examine the test failure(s) this week.
Cheers
On Thu, Jan 2, 2014 at 10:58 AM, Andrew Purtell <ap...@apache.org> wrote:
> On Thu, Jan 2, 2014 at 10:52 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > The TestSplitLogWorker failures were all under JDK 6 on Ubuntu 12. See
> > HBASE-10268.
>
>
> This is correct. There is a past-o in the table for JDK7 - the second item
> should be org.apache.hadoop.hbase.TestIOFencing.
>
>
> On Thu, Jan 2, 2014 at 10:52 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > I have finished 50 runs of the unit test suite using JDK 6 and JDK 7 on
> > RHEL 6, Ubuntu 12, and Ubuntu 13. The combined results by JDK version are
> > as follows:
> >
> > JDK 6 - Java(TM) SE Runtime Environment (build 1.6.0_43-b01) HotSpot(TM)
> > 64-Bit Server VM (build 20.14-b01, mixed mode)
> >
> >
> > Count Test
> >
> >
> >
> >
> >
> >
> > 5 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
> >
> >
> >
> >
> >
> >
> > 3 org.apache.hadoop.hbase.TestIOFencing
> >
> >
> >
> >
> >
> >
> > 2 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
> >
> >
> >
> >
> >
> >
> > 1
> >
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
> >
> >
> >
> >
> >
> >
> > 1 org.apache.hadoop.hbase.TestNodeHealthCheckChore
> >
> >
> >
> >
> >
> >
> > 1 org.apache.hadoop.hbase.procedure.TestZKProcedure
> >
> >
> >
> >
> >
> >
> > 1 org.apache.hadoop.hbase.client.TestMultiParallel
> >
> >
> >
> >
> >
> >
> > JDK 7 - Java(TM) SE Runtime Environment (build 1.7.0_21-b11) HotSpot(TM)
> > 64-Bit Server VM (build 23.21-b01, mixed mode)
> >
> > Count Test
> >
> >
> >
> > 1 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
> >
> >
> >
> > 1 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
> >
> >
> >
> >
> > I opened HBASE-10267 for the TestNamespaceCommands failures and will work
> > on this since I bet it is related to recent changes I made in this area.
> >
> > The TestSplitLogWorker failures were all under JDK 6 on Ubuntu 12. See
> > HBASE-10268.
> >
> > I found five related issues for TestIOFencing failures: HBASE-8520,
> > HBASE-8646, HBASE-8889, HBASE-8907, and HBASE-9023. Therefore I am
> inclined
> > to disable this test on 0.98 branch for us to sort out on trunk later
> since
> > it's taking a while to get it right.
> >
> >
> > On Sun, Dec 29, 2013 at 3:59 PM, Andrew Purtell <apurtell@apache.org
> >wrote:
> >
> >> Next week I would like to roll the first 0.98.0 RC. Let's see.
> >>
> >> Some remaining work needs committing. Then, will work up a Google Docs
> >> spreadsheet for package checks, unit test suite runs, and cluster
> testing
> >> with hbase-it, LoadTestTool, and YCSB, and fill it in.
> >>
> >> The unit test suite is looking "ok". There are occasional failures on
> >> builds.apache.org, but these may have system load as the underlying
> >> cause. Unit test suite runs on more well endowed EC2 VMs are passing
> >> consistently. Let me add the caveat that I have so far not completed
> >> running through the full VM-OS matrix:
> >>
> >> VM:
> >>
> >>
> >> - Oracle JDK 6
> >> - Oracle JDK 7
> >> - OpenJDK 7
> >> - OpenJDK 8*
> >>
> >> OS (unit tests):
> >>
> >>
> >> - Amazon Linux
> >> - RHEL 6.4
> >> - Ubuntu Server 12.04.3 LTS
> >> - Ubuntu Server 13
> >>
> >> OS (cluster);
> >>
> >>
> >> - Amazon Linux
> >> - RHEL 6.5
> >>
> >> If I identify repeatable test failures under a particular combination,
> >> then I'll open a blocker JIRA for it. Regarding failures on
> >> builds.apache.org, if there are JIRAs filed for these we should
> >> certainly take a look but I suggest (and personally will be inclined to)
> >> weighting more highly evidence gathered from testing elsewhere.
> >>
> >> * - Java 8 tests are just for my own curiosity. Actually, you may be
> >> pleasantly surprised to know that all unit tests for HBase 0.98 and
> trunk
> >> were consistently passing on a Java 8 build a few weeks ago.
> >>
> >> --
> >> Best regards,
> >>
> >> - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: Next week I would like to roll the first 0.98.0 RC
Posted by Andrew Purtell <ap...@apache.org>.
On Thu, Jan 2, 2014 at 10:52 AM, Andrew Purtell <ap...@apache.org> wrote:
> The TestSplitLogWorker failures were all under JDK 6 on Ubuntu 12. See
> HBASE-10268.
This is correct. There is a past-o in the table for JDK7 - the second item
should be org.apache.hadoop.hbase.TestIOFencing.
On Thu, Jan 2, 2014 at 10:52 AM, Andrew Purtell <ap...@apache.org> wrote:
> I have finished 50 runs of the unit test suite using JDK 6 and JDK 7 on
> RHEL 6, Ubuntu 12, and Ubuntu 13. The combined results by JDK version are
> as follows:
>
> JDK 6 - Java(TM) SE Runtime Environment (build 1.6.0_43-b01) HotSpot(TM)
> 64-Bit Server VM (build 20.14-b01, mixed mode)
>
>
> Count Test
>
>
>
>
>
>
> 5 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
>
>
>
>
>
>
> 3 org.apache.hadoop.hbase.TestIOFencing
>
>
>
>
>
>
> 2 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
>
>
>
>
>
>
> 1
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
>
>
>
>
>
>
> 1 org.apache.hadoop.hbase.TestNodeHealthCheckChore
>
>
>
>
>
>
> 1 org.apache.hadoop.hbase.procedure.TestZKProcedure
>
>
>
>
>
>
> 1 org.apache.hadoop.hbase.client.TestMultiParallel
>
>
>
>
>
>
> JDK 7 - Java(TM) SE Runtime Environment (build 1.7.0_21-b11) HotSpot(TM)
> 64-Bit Server VM (build 23.21-b01, mixed mode)
>
> Count Test
>
>
>
> 1 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
>
>
>
> 1 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
>
>
>
>
> I opened HBASE-10267 for the TestNamespaceCommands failures and will work
> on this since I bet it is related to recent changes I made in this area.
>
> The TestSplitLogWorker failures were all under JDK 6 on Ubuntu 12. See
> HBASE-10268.
>
> I found five related issues for TestIOFencing failures: HBASE-8520,
> HBASE-8646, HBASE-8889, HBASE-8907, and HBASE-9023. Therefore I am inclined
> to disable this test on 0.98 branch for us to sort out on trunk later since
> it's taking a while to get it right.
>
>
> On Sun, Dec 29, 2013 at 3:59 PM, Andrew Purtell <ap...@apache.org>wrote:
>
>> Next week I would like to roll the first 0.98.0 RC. Let's see.
>>
>> Some remaining work needs committing. Then, will work up a Google Docs
>> spreadsheet for package checks, unit test suite runs, and cluster testing
>> with hbase-it, LoadTestTool, and YCSB, and fill it in.
>>
>> The unit test suite is looking "ok". There are occasional failures on
>> builds.apache.org, but these may have system load as the underlying
>> cause. Unit test suite runs on more well endowed EC2 VMs are passing
>> consistently. Let me add the caveat that I have so far not completed
>> running through the full VM-OS matrix:
>>
>> VM:
>>
>>
>> - Oracle JDK 6
>> - Oracle JDK 7
>> - OpenJDK 7
>> - OpenJDK 8*
>>
>> OS (unit tests):
>>
>>
>> - Amazon Linux
>> - RHEL 6.4
>> - Ubuntu Server 12.04.3 LTS
>> - Ubuntu Server 13
>>
>> OS (cluster);
>>
>>
>> - Amazon Linux
>> - RHEL 6.5
>>
>> If I identify repeatable test failures under a particular combination,
>> then I'll open a blocker JIRA for it. Regarding failures on
>> builds.apache.org, if there are JIRAs filed for these we should
>> certainly take a look but I suggest (and personally will be inclined to)
>> weighting more highly evidence gathered from testing elsewhere.
>>
>> * - Java 8 tests are just for my own curiosity. Actually, you may be
>> pleasantly surprised to know that all unit tests for HBase 0.98 and trunk
>> were consistently passing on a Java 8 build a few weeks ago.
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Next week I would like to roll the first 0.98.0 RC
Posted by Andrew Purtell <ap...@apache.org>.
On Thu, Jan 2, 2014 at 11:11 AM, Stack <st...@duboce.net> wrote:
> > 5 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
>
> This one has been flakey for ever. Is it always the same test that fails?
> Is that available to you Andrew?
>
No, all target/ directories were rewritten each run.
> > 3 org.apache.hadoop.hbase.TestIOFencing
>
> Ok on disabling but we need to fix this one... so blocker on a 1.0.0?
If I have time I will see if trying one of the existing issues a bit harder
is enough, but it makes sense to me to disable it and make a blocker for
1.0 if this test becomes a time sink.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Next week I would like to roll the first 0.98.0 RC
Posted by Stack <st...@duboce.net>.
On Thu, Jan 2, 2014 at 10:52 AM, Andrew Purtell <ap...@apache.org> wrote:
> I have finished 50 runs of the unit test suite using JDK 6 and JDK 7 on
> RHEL 6, Ubuntu 12, and Ubuntu 13. The combined results by JDK version are
> as follows:
>
> JDK 6 - Java(TM) SE Runtime Environment (build 1.6.0_43-b01) HotSpot(TM)
> 64-Bit Server VM (build 20.14-b01, mixed mode)
>
>
> Count Test
>
>
>
>
>
>
> 5 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
>
>
>
>
This one has been flakey for ever. Is it always the same test that fails?
Is that available to you Andrew?
>
>
>
> 3 org.apache.hadoop.hbase.TestIOFencing
>
>
>
Ok on disabling but we need to fix this one... so blocker on a 1.0.0?
St.Ack
Re: Next week I would like to roll the first 0.98.0 RC
Posted by Andrew Purtell <ap...@apache.org>.
On Thu, Jan 2, 2014 at 11:08 AM, lars hofhansl <la...@apache.org> wrote:
> Do you need some help, Andy?
>
> I can look at some of these.
>
Great! If you could look at HBASE-10268 that would be very helpful.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Next week I would like to roll the first 0.98.0 RC
Posted by lars hofhansl <la...@apache.org>.
Do you need some help, Andy?
I can look at some of these.
-- Lars
________________________________
From: Andrew Purtell <ap...@apache.org>
To: "dev@hbase.apache.org" <de...@hbase.apache.org>
Sent: Thursday, January 2, 2014 10:52 AM
Subject: Re: Next week I would like to roll the first 0.98.0 RC
I have finished 50 runs of the unit test suite using JDK 6 and JDK 7 on
RHEL 6, Ubuntu 12, and Ubuntu 13. The combined results by JDK version are
as follows:
JDK 6 - Java(TM) SE Runtime Environment (build 1.6.0_43-b01) HotSpot(TM)
64-Bit Server VM (build 20.14-b01, mixed mode)
Count Test
5 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
3 org.apache.hadoop.hbase.TestIOFencing
2 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
1
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
1 org.apache.hadoop.hbase.TestNodeHealthCheckChore
1 org.apache.hadoop.hbase.procedure.TestZKProcedure
1 org.apache.hadoop.hbase.client.TestMultiParallel
JDK 7 - Java(TM) SE Runtime Environment (build 1.7.0_21-b11) HotSpot(TM)
64-Bit Server VM (build 23.21-b01, mixed mode)
Count Test
1 org.apache.hadoop.hbase.security.access.TestNamespaceCommands
1 org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
I opened HBASE-10267 for the TestNamespaceCommands failures and will work
on this since I bet it is related to recent changes I made in this area.
The TestSplitLogWorker failures were all under JDK 6 on Ubuntu 12. See
HBASE-10268.
I found five related issues for TestIOFencing failures: HBASE-8520,
HBASE-8646, HBASE-8889, HBASE-8907, and HBASE-9023. Therefore I am inclined
to disable this test on 0.98 branch for us to sort out on trunk later since
it's taking a while to get it right.
On Sun, Dec 29, 2013 at 3:59 PM, Andrew Purtell <ap...@apache.org> wrote:
> Next week I would like to roll the first 0.98.0 RC. Let's see.
>
> Some remaining work needs committing. Then, will work up a Google Docs
> spreadsheet for package checks, unit test suite runs, and cluster testing
> with hbase-it, LoadTestTool, and YCSB, and fill it in.
>
> The unit test suite is looking "ok". There are occasional failures on
> builds.apache.org, but these may have system load as the underlying
> cause. Unit test suite runs on more well endowed EC2 VMs are passing
> consistently. Let me add the caveat that I have so far not completed
> running through the full VM-OS matrix:
>
> VM:
>
>
> - Oracle JDK 6
> - Oracle JDK 7
> - OpenJDK 7
> - OpenJDK 8*
>
> OS (unit tests):
>
>
> - Amazon Linux
> - RHEL 6.4
> - Ubuntu Server 12.04.3 LTS
> - Ubuntu Server 13
>
> OS (cluster);
>
>
> - Amazon Linux
> - RHEL 6.5
>
> If I identify repeatable test failures under a particular combination,
> then I'll open a blocker JIRA for it. Regarding failures on
> builds.apache.org, if there are JIRAs filed for these we should certainly
> take a look but I suggest (and personally will be inclined to) weighting
> more highly evidence gathered from testing elsewhere.
>
> * - Java 8 tests are just for my own curiosity. Actually, you may be
> pleasantly surprised to know that all unit tests for HBase 0.98 and trunk
> were consistently passing on a Java 8 build a few weeks ago.
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)