You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Michael Ho (JIRA)" <ji...@apache.org> on 2018/08/21 06:19:00 UTC
[jira] [Commented] (IMPALA-7061) PlannerTests should assign HBase
splits as part of the test
[ https://issues.apache.org/jira/browse/IMPALA-7061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586987#comment-16586987 ]
Michael Ho commented on IMPALA-7061:
------------------------------------
Hi [~joemcdonnell], I just hit the following again during GVO (https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3020/testReport/junit/org.apache.impala.planner/S3PlannerTest/org_apache_impala_planner_S3PlannerTest/). Is this supposed to be fixed by this JIRA already or should I open a new JIRA ?
{noformat}
org.apache.impala.planner.S3PlannerTest.org.apache.impala.planner.S3PlannerTest
Failed to assign regions to servers after 60000 millis.
Stacktrace
java.lang.IllegalStateException: Failed to assign regions to servers after 60000 millis.
at org.apache.impala.datagenerator.HBaseTestDataRegionAssignment.performAssignment(HBaseTestDataRegionAssignment.java:153)
at org.apache.impala.planner.PlannerTestBase.setUp(PlannerTestBase.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
{noformat}
> PlannerTests should assign HBase splits as part of the test
> -----------------------------------------------------------
>
> Key: IMPALA-7061
> URL: https://issues.apache.org/jira/browse/IMPALA-7061
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend, Infrastructure
> Affects Versions: Impala 2.13.0, Impala 3.1.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> Some frontend PlannerTests rely on HBase tables being split into specific regions with those regions assigned to specific region servers. Right now, the HBase tables are created via the HBase shell with a single region, then they are populated via Hive DMLs, then there is a Java program that splits the tables into appropriate regions and assigns those regions to region servers. Once that is done, nothing is maintaining the assignments. The Java code for doing the splitting is hard to maintain and flaky. The assignments can sometimes drift due to rebalancing.
> We should convert this to specify the splits at HBase table creation time. We should have the frontend PlannerTest do assignments at setup time to avoid flakiness due to rebalancing. This should move some flakiness out of dataload.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org