You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Norbert Kalmar <nk...@apache.org> on 2019/07/10 14:50:01 UTC
Integration Test - and running them
Hi HBase devs,
I've been trying to implement integration test for Bulkload replication.
I'm using the already existing IntegrationTestBulkLoad as a base.
My problem is actually running these tests.
If I try to run them (tried multiple tests like Replication, BigLinkedList)
with maven failsafe, I always get JVM outofmemory error every time it tries
to create the mini cluster. I also tripled the default 4GB heap size. Looks
like it's spanning few thousand threads, which explains the heap running
out of memory.
Ultimately the whole test fails with a one liner saying:
IntegrationTestBigLinkedList>IntegrationTestBase.setUp:170->setUpCluster:1840
» YarnRuntime
I might have missed something. I also tried to run against an existing
cluster, but had some trouble there either.
In both cases, I followed the official HBase guide available at
https://hbase.apache.org/book.html#hbase.tests
I also found a few open issues just on Bulkload integration test:
https://issues.apache.org/jira/browse/HBASE-12860
https://issues.apache.org/jira/browse/HBASE-14846
Is there something trivial I am missing? Does someone run these integration
tests? If so, what is the key here? I could up the memory and timeout I
guess, but I feel like I might be missing something here.
Thanks!
Norbert
Re: Integration Test - and running them
Posted by Norbert Kalmar <nk...@cloudera.com.INVALID>.
Thanks for the reply. I created a jira (HBASE-22678)
My MiniCluster logs are empty.
I added some extra info to the jira.
Regards,
Norbert
On Thu, Jul 11, 2019 at 4:58 AM OpenInx <op...@gmail.com> wrote:
> I run the integrate test under my host, and also get the failure. I think
> it may worth to
> file a seperate jira to track the problem.
>
> [INFO] -------------------------------------------------------
> [INFO] Running org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 55.137 s <<< FAILURE! - in
> org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad
> [ERROR]
> testBulkLoad(org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad)
> Time elapsed: 54.469 s <<< ERROR!
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.io.IOException: ResourceManager failed to start. Final state is
> STOPPED
> at
>
> org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.setUpCluster(IntegrationTestBulkLoad.java:764)
> Caused by: java.io.IOException: ResourceManager failed to start. Final
> state is STOPPED
> at
>
> org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.setUpCluster(IntegrationTestBulkLoad.java:764)
>
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR]
> IntegrationTestBulkLoad>IntegrationTestBase.setUp:170->setUpCluster:764 »
> YarnRuntime
> [INFO]
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
>
> On Thu, Jul 11, 2019 at 10:51 AM OpenInx <op...@gmail.com> wrote:
>
> > Hi Norbert Kalmar
> >
> > You mean run the test by the command: mvn test
> > -Dtest=IntegrationTestBigLinkedList ?
> > Then I think it should have some detail logs under your directory:
> > <hbase-project>/hbase-it/target/surefire-reports
> >
> > Aslo there should be some logs under you hbase-it/target/MiniCluster-xxx
> > directory...
> >
> > On Wed, Jul 10, 2019 at 10:50 PM Norbert Kalmar <nk...@apache.org>
> > wrote:
> >
> >> Hi HBase devs,
> >>
> >> I've been trying to implement integration test for Bulkload replication.
> >> I'm using the already existing IntegrationTestBulkLoad as a base.
> >> My problem is actually running these tests.
> >> If I try to run them (tried multiple tests like Replication,
> >> BigLinkedList)
> >> with maven failsafe, I always get JVM outofmemory error every time it
> >> tries
> >> to create the mini cluster. I also tripled the default 4GB heap size.
> >> Looks
> >> like it's spanning few thousand threads, which explains the heap running
> >> out of memory.
> >>
> >> Ultimately the whole test fails with a one liner saying:
> >>
> >>
> IntegrationTestBigLinkedList>IntegrationTestBase.setUp:170->setUpCluster:1840
> >> » YarnRuntime
> >>
> >> I might have missed something. I also tried to run against an existing
> >> cluster, but had some trouble there either.
> >>
> >> In both cases, I followed the official HBase guide available at
> >> https://hbase.apache.org/book.html#hbase.tests
> >>
> >> I also found a few open issues just on Bulkload integration test:
> >> https://issues.apache.org/jira/browse/HBASE-12860
> >> https://issues.apache.org/jira/browse/HBASE-14846
> >>
> >> Is there something trivial I am missing? Does someone run these
> >> integration
> >> tests? If so, what is the key here? I could up the memory and timeout I
> >> guess, but I feel like I might be missing something here.
> >>
> >> Thanks!
> >> Norbert
> >>
> >
>
Re: Integration Test - and running them
Posted by OpenInx <op...@gmail.com>.
I run the integrate test under my host, and also get the failure. I think
it may worth to
file a seperate jira to track the problem.
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
55.137 s <<< FAILURE! - in
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad
[ERROR]
testBulkLoad(org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad)
Time elapsed: 54.469 s <<< ERROR!
org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.io.IOException: ResourceManager failed to start. Final state is STOPPED
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.setUpCluster(IntegrationTestBulkLoad.java:764)
Caused by: java.io.IOException: ResourceManager failed to start. Final
state is STOPPED
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.setUpCluster(IntegrationTestBulkLoad.java:764)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]
IntegrationTestBulkLoad>IntegrationTestBase.setUp:170->setUpCluster:764 »
YarnRuntime
[INFO]
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
On Thu, Jul 11, 2019 at 10:51 AM OpenInx <op...@gmail.com> wrote:
> Hi Norbert Kalmar
>
> You mean run the test by the command: mvn test
> -Dtest=IntegrationTestBigLinkedList ?
> Then I think it should have some detail logs under your directory:
> <hbase-project>/hbase-it/target/surefire-reports
>
> Aslo there should be some logs under you hbase-it/target/MiniCluster-xxx
> directory...
>
> On Wed, Jul 10, 2019 at 10:50 PM Norbert Kalmar <nk...@apache.org>
> wrote:
>
>> Hi HBase devs,
>>
>> I've been trying to implement integration test for Bulkload replication.
>> I'm using the already existing IntegrationTestBulkLoad as a base.
>> My problem is actually running these tests.
>> If I try to run them (tried multiple tests like Replication,
>> BigLinkedList)
>> with maven failsafe, I always get JVM outofmemory error every time it
>> tries
>> to create the mini cluster. I also tripled the default 4GB heap size.
>> Looks
>> like it's spanning few thousand threads, which explains the heap running
>> out of memory.
>>
>> Ultimately the whole test fails with a one liner saying:
>>
>> IntegrationTestBigLinkedList>IntegrationTestBase.setUp:170->setUpCluster:1840
>> » YarnRuntime
>>
>> I might have missed something. I also tried to run against an existing
>> cluster, but had some trouble there either.
>>
>> In both cases, I followed the official HBase guide available at
>> https://hbase.apache.org/book.html#hbase.tests
>>
>> I also found a few open issues just on Bulkload integration test:
>> https://issues.apache.org/jira/browse/HBASE-12860
>> https://issues.apache.org/jira/browse/HBASE-14846
>>
>> Is there something trivial I am missing? Does someone run these
>> integration
>> tests? If so, what is the key here? I could up the memory and timeout I
>> guess, but I feel like I might be missing something here.
>>
>> Thanks!
>> Norbert
>>
>
Re: Integration Test - and running them
Posted by OpenInx <op...@gmail.com>.
Hi Norbert Kalmar
You mean run the test by the command: mvn test
-Dtest=IntegrationTestBigLinkedList ?
Then I think it should have some detail logs under your directory:
<hbase-project>/hbase-it/target/surefire-reports
Aslo there should be some logs under you hbase-it/target/MiniCluster-xxx
directory...
On Wed, Jul 10, 2019 at 10:50 PM Norbert Kalmar <nk...@apache.org> wrote:
> Hi HBase devs,
>
> I've been trying to implement integration test for Bulkload replication.
> I'm using the already existing IntegrationTestBulkLoad as a base.
> My problem is actually running these tests.
> If I try to run them (tried multiple tests like Replication, BigLinkedList)
> with maven failsafe, I always get JVM outofmemory error every time it tries
> to create the mini cluster. I also tripled the default 4GB heap size. Looks
> like it's spanning few thousand threads, which explains the heap running
> out of memory.
>
> Ultimately the whole test fails with a one liner saying:
>
> IntegrationTestBigLinkedList>IntegrationTestBase.setUp:170->setUpCluster:1840
> » YarnRuntime
>
> I might have missed something. I also tried to run against an existing
> cluster, but had some trouble there either.
>
> In both cases, I followed the official HBase guide available at
> https://hbase.apache.org/book.html#hbase.tests
>
> I also found a few open issues just on Bulkload integration test:
> https://issues.apache.org/jira/browse/HBASE-12860
> https://issues.apache.org/jira/browse/HBASE-14846
>
> Is there something trivial I am missing? Does someone run these integration
> tests? If so, what is the key here? I could up the memory and timeout I
> guess, but I feel like I might be missing something here.
>
> Thanks!
> Norbert
>