You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Deepak Jaiswal <dj...@hortonworks.com> on 2018/06/27 17:13:17 UTC

Unstable Hive QA

Hi All,

It seems we are going back to instability in Hive QA runs. In the past few days I saw many runs where the failures were completely independent. When those tests are run locally, they don’t fail which makes them harder to catch.

On one side I think having green run to commit makes sense, however, on the other side, the development is unnecessarily blocked. Putting the randomly failing tests in disabled list is also not a good idea as it brings down the code coverage.
Any suggestions?

Regards,
Deepak

Re: Unstable Hive QA

Posted by Deepak Jaiswal <dj...@hortonworks.com>.
This is most likely due to cleanup not happening properly from the previous runs.

On 6/28/18, 10:48 AM, "Vineet Garg" <vg...@hortonworks.com> wrote:

    I have also bumped into flaky druid failures although with different symptoms (Wait time exhausted and we have [1] out of [1] segments not loaded yet)
    
    All of the druid failures in this case are during create table with following error stack: (Not sure why this is intermittent)
    
    org.apache.hadoop.security.ShellBasedUnixGroupsMapping$PartialGroupNameException: The user name 'hive_test_user' is not found. id: hive_test_user: no such user
    id: hive_test_user: no such user
    
    at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:294) ~[hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:207) [hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:97) [hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:51) [hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:384) [hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319) [hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269) [hadoop-common-3.1.0.jar:?]
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542) [guava-19.0.jar:?]
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323) [guava-19.0.jar:?]
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286) [guava-19.0.jar:?]
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201) [guava-19.0.jar:?]
    at com.google.common.cache.LocalCache.get(LocalCache.java:3953) [guava-19.0.jar:?]
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957) [guava-19.0.jar:?]
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875) [guava-19.0.jar:?]
    at org.apache.hadoop.security.Groups.getGroups(Groups.java:227) [hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.security.UserGroupInformation.getGroups(UserGroupInformation.java:1540) [hadoop-common-3.1.0.jar:?]
    at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2689) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2341) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2012) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1712) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1706) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:218) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) [hive-cli-4.0.0-SNAPSHOT.jar:?]
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188) [hive-cli-4.0.0-SNAPSHOT.jar:?]
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) [hive-cli-4.0.0-SNAPSHOT.jar:?]
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335) [hive-cli-4.0.0-SNAPSHOT.jar:?]
    at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1340) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1314) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:171) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver(TestMiniDruidCliDriver.java:59) [test-classes/:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_102]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_102]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_102]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) [junit-4.11.jar:?]
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.11.jar:?]
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) [junit-4.11.jar:?]
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) [junit-4.11.jar:?]
    at org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) [junit-4.11.jar:?]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) [junit-4.11.jar:?]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309) [junit-4.11.jar:?]
    at org.junit.runners.Suite.runChild(Suite.java:127) [junit-4.11.jar:?]
    at org.junit.runners.Suite.runChild(Suite.java:26) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) [junit-4.11.jar:?]
    at org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:73) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
    at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.11.jar:?]
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309) [junit-4.11.jar:?]
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) [surefire-junit4-2.21.0.jar:2.21.0]
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) [surefire-junit4-2.21.0.jar:2.21.0]
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) [surefire-junit4-2.21.0.jar:2.21.0]
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) [surefire-junit4-2.21.0.jar:2.21.0]
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379) [surefire-booter-2.21.0.jar:2.21.0]
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340) [surefire-booter-2.21.0.jar:2.21.0]
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) [surefire-booter-2.21.0.jar:2.21.0]
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413) [surefire-booter-2.21.0.jar:2.21.0]
    2018-06-27T22:24:51,363  INFO [d00e737b-0dde-4230-ae29-d20498bf8332 main] ql.Context: New scratch dir is hdfs://localhost:37593/home/hiveptest
    
    
    Vineet
    
    On Jun 27, 2018, at 11:47 PM, Deepak Jaiswal <dj...@hortonworks.com>> wrote:
    
    Ptests have become really unstable.
    
    The druid tests are failing randomly,
    https://builds.apache.org/job/PreCommit-HIVE-Build/12203/testReport
    
    Should we disable them?
    
    Deepak
    
    On 6/27/18, 10:13 AM, "Deepak Jaiswal" <dj...@hortonworks.com> wrote:
    
       Hi All,
    
       It seems we are going back to instability in Hive QA runs. In the past few days I saw many runs where the failures were completely independent. When those tests are run locally, they don’t fail which makes them harder to catch.
    
       On one side I think having green run to commit makes sense, however, on the other side, the development is unnecessarily blocked. Putting the randomly failing tests in disabled list is also not a good idea as it brings down the code coverage.
       Any suggestions?
    
       Regards,
       Deepak

Re: Unstable Hive QA

Posted by Vineet Garg <vg...@hortonworks.com>.
I have also bumped into flaky druid failures although with different symptoms (Wait time exhausted and we have [1] out of [1] segments not loaded yet)

All of the druid failures in this case are during create table with following error stack: (Not sure why this is intermittent)

org.apache.hadoop.security.ShellBasedUnixGroupsMapping$PartialGroupNameException: The user name 'hive_test_user' is not found. id: hive_test_user: no such user
id: hive_test_user: no such user

at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:294) ~[hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:207) [hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:97) [hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:51) [hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:384) [hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319) [hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269) [hadoop-common-3.1.0.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542) [guava-19.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323) [guava-19.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286) [guava-19.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201) [guava-19.0.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3953) [guava-19.0.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957) [guava-19.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875) [guava-19.0.jar:?]
at org.apache.hadoop.security.Groups.getGroups(Groups.java:227) [hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.security.UserGroupInformation.getGroups(UserGroupInformation.java:1540) [hadoop-common-3.1.0.jar:?]
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2689) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2341) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2012) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1712) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1706) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:218) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) [hive-cli-4.0.0-SNAPSHOT.jar:?]
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188) [hive-cli-4.0.0-SNAPSHOT.jar:?]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) [hive-cli-4.0.0-SNAPSHOT.jar:?]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335) [hive-cli-4.0.0-SNAPSHOT.jar:?]
at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1340) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1314) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:171) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver(TestMiniDruidCliDriver.java:59) [test-classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_102]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_102]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) [junit-4.11.jar:?]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.11.jar:?]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) [junit-4.11.jar:?]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) [junit-4.11.jar:?]
at org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) [junit-4.11.jar:?]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) [junit-4.11.jar:?]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner.run(ParentRunner.java:309) [junit-4.11.jar:?]
at org.junit.runners.Suite.runChild(Suite.java:127) [junit-4.11.jar:?]
at org.junit.runners.Suite.runChild(Suite.java:26) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) [junit-4.11.jar:?]
at org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:73) [hive-it-util-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.11.jar:?]
at org.junit.runners.ParentRunner.run(ParentRunner.java:309) [junit-4.11.jar:?]
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) [surefire-junit4-2.21.0.jar:2.21.0]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) [surefire-junit4-2.21.0.jar:2.21.0]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) [surefire-junit4-2.21.0.jar:2.21.0]
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) [surefire-junit4-2.21.0.jar:2.21.0]
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379) [surefire-booter-2.21.0.jar:2.21.0]
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340) [surefire-booter-2.21.0.jar:2.21.0]
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) [surefire-booter-2.21.0.jar:2.21.0]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413) [surefire-booter-2.21.0.jar:2.21.0]
2018-06-27T22:24:51,363  INFO [d00e737b-0dde-4230-ae29-d20498bf8332 main] ql.Context: New scratch dir is hdfs://localhost:37593/home/hiveptest


Vineet

On Jun 27, 2018, at 11:47 PM, Deepak Jaiswal <dj...@hortonworks.com>> wrote:

Ptests have become really unstable.

The druid tests are failing randomly,
https://builds.apache.org/job/PreCommit-HIVE-Build/12203/testReport

Should we disable them?

Deepak

On 6/27/18, 10:13 AM, "Deepak Jaiswal" <dj...@hortonworks.com> wrote:

   Hi All,

   It seems we are going back to instability in Hive QA runs. In the past few days I saw many runs where the failures were completely independent. When those tests are run locally, they don’t fail which makes them harder to catch.

   On one side I think having green run to commit makes sense, however, on the other side, the development is unnecessarily blocked. Putting the randomly failing tests in disabled list is also not a good idea as it brings down the code coverage.
   Any suggestions?

   Regards,
   Deepak

Re: Unstable Hive QA

Posted by Deepak Jaiswal <dj...@hortonworks.com>.
Ptests have become really unstable.

The druid tests are failing randomly,
https://builds.apache.org/job/PreCommit-HIVE-Build/12203/testReport

Should we disable them?

Deepak

On 6/27/18, 10:13 AM, "Deepak Jaiswal" <dj...@hortonworks.com> wrote:

    Hi All,
    
    It seems we are going back to instability in Hive QA runs. In the past few days I saw many runs where the failures were completely independent. When those tests are run locally, they don’t fail which makes them harder to catch.
    
    On one side I think having green run to commit makes sense, however, on the other side, the development is unnecessarily blocked. Putting the randomly failing tests in disabled list is also not a good idea as it brings down the code coverage.
    Any suggestions?
    
    Regards,
    Deepak