You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/04/08 01:56:17 UTC
[GitHub] [incubator-pinot] daniellavoie opened a new issue #6757: Test failure on Pinot Hadoop
daniellavoie opened a new issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757
Our internal CI system picked up regressions on commit `4b0db5a3cfdfe69551cef235f9d8c25ffe31b254` found on `master`. I was able to reproduced locally with a fresh maven repository.
```
ERROR] Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 2.482 s <<< FAILURE! - in org.apache.pinot.hadoop.io.PinotOutputFormatTest
[ERROR] org.apache.pinot.hadoop.io.PinotOutputFormatTest.testPinotOutputFormat Time elapsed: 1.836 s <<< FAILURE!
java.lang.NoSuchMethodError: com.google.common.cache.CacheBuilder.ticker(Lcom/google/common/base/Ticker;)Lcom/google/common/cache/CacheBuilder;
at org.apache.hadoop.security.Groups.<init>(Groups.java:99)
at org.apache.hadoop.security.Groups.<init>(Groups.java:74)
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:303)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:790)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:760)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:633)
at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:72)
at org.apache.hadoop.mapreduce.Job.<init>(Job.java:142)
at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:185)
at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:165)
at org.apache.pinot.hadoop.io.PinotOutputFormatTest.testPinotOutputFormat(PinotOutputFormatTest.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:661)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:869)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1193)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:126)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.privateRun(TestRunner.java:744)
at org.testng.TestRunner.run(TestRunner.java:602)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
at org.testng.SuiteRunner.run(SuiteRunner.java:289)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1226)
at org.testng.TestNG.runSuites(TestNG.java:1144)
at org.testng.TestNG.run(TestNG.java:1115)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:136)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:145)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
[ERROR] org.apache.pinot.hadoop.io.PinotOutputFormatTest.tearDown Time elapsed: 1.837 s <<< FAILURE!
java.io.FileNotFoundException: File does not exist: /var/folders/mx/slpjv4nd3713gnsddc6yyb_c0000gn/T/PinotOutputFormatTest
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2275)
at org.apache.pinot.hadoop.io.PinotOutputFormatTest.tearDown(PinotOutputFormatTest.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:523)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:224)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:146)
at org.testng.internal.TestMethodWorker.invokeAfterClassMethods(TestMethodWorker.java:212)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
at org.testng.TestRunner.privateRun(TestRunner.java:744)
at org.testng.TestRunner.run(TestRunner.java:602)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
at org.testng.SuiteRunner.run(SuiteRunner.java:289)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1226)
at org.testng.TestNG.runSuites(TestNG.java:1144)
at org.testng.TestNG.run(TestNG.java:1115)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:136)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:145)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] daniellavoie removed a comment on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
daniellavoie removed a comment on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815840218
Found this:
```
$ MAVEN_OPTS="-verbose" mvn package| grep com.google.common.cache.CacheBuilder
[Loaded com.google.common.cache.CacheBuilder from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$3 from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$1 from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$2 from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$OneWeigher from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$NullListener from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
```
Wondering if the surefire plugin is forking the tests in a distinct JVM. Sounds like it's not the case and would explain why we are getting different behavior from the IDE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] daniellavoie commented on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
daniellavoie commented on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815840218
Found this:
```
$ MAVEN_OPTS="-verbose" mvn package| grep com.google.common.cache.CacheBuilder
[Loaded com.google.common.cache.CacheBuilder from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$3 from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$1 from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$2 from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$OneWeigher from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
[Loaded com.google.common.cache.CacheBuilder$NullListener from file:/Applications/apache-maven-3.6.3/lib/guava-25.1-android.jar]
```
Wondering if the surefire plugin is forking the tests in a distinct JVM. Sounds like it's not the case and would explain why we are getting different behavior from the IDE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] daniellavoie commented on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
daniellavoie commented on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815895589
I added `-verbose` to the surefire plugin and found the following interesting trace
```
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-verbose</argLine>
</configuration>
</plugin>
```
```
'[Loaded com.google.common.cache.CacheBuilder from file:/Users/daniellavoie/.m2/repository/org/apache/pinot/pinot-common/0.7.0-SNAPSHOT/pinot-common-0.7.0-SNAPSHOT-shaded.jar]'.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] daniellavoie commented on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
daniellavoie commented on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815816132
Might be caused by the shade plugin configuration.
```
<plugin>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<!--
Usually in hadoop environment, there are multiple jars with different versions.
Most of the NoSuchMethodExceptions are caused by class loading conflicts.
Class relocation ensures the reference of certain packages/classes in Pinot code to
shaded libs, e.g. jackson or guava.
Ref: https://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html
-->
<relocations>
<relocation>
<pattern>com.google.common.base</pattern>
<shadedPattern>shaded.com.google.common.base</shadedPattern>
</relocation>
<relocation>
<pattern>com.fasterxml.jackson</pattern>
<shadedPattern>shaded.com.fasterxml.jackson</shadedPattern>
</relocation>
</relocations>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.apache.pinot.hadoop.PinotHadoopJobLauncher</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] daniellavoie commented on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
daniellavoie commented on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815814309
Seems to run fine from the IDE, but still getting the error with `mvn test`
```
pinot-hadoop % mvn dependency:tree -Dverbose -Dincludes=com.google.guava
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.16
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 16
[INFO] os.detected.classifier: osx-x86_64
[INFO]
[INFO] -------------------< org.apache.pinot:pinot-hadoop >--------------------
[INFO] Building Pinot Hadoop 0.7.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ pinot-hadoop ---
[INFO] org.apache.pinot:pinot-hadoop:jar:0.7.0-SNAPSHOT
[INFO] +- org.apache.pinot:pinot-common:jar:shaded:0.7.0-SNAPSHOT:compile
[INFO] | +- org.apache.calcite:calcite-core:jar:1.19.0:compile
[INFO] | | +- org.apache.calcite:calcite-linq4j:jar:1.19.0:compile
[INFO] | | | \- (com.google.guava:guava:jar:20.0:compile - version managed from 19.0; omitted for duplicate)
[INFO] | | \- (com.google.guava:guava:jar:20.0:compile - version managed from 19.0; omitted for duplicate)
[INFO] | +- org.apache.calcite:calcite-babel:jar:1.19.0:compile
[INFO] | | \- (com.google.guava:guava:jar:20.0:compile - version managed from 19.0; omitted for duplicate)
[INFO] | +- com.google.guava:guava:jar:20.0:compile
[INFO] | +- org.apache.helix:helix-core:jar:0.9.8:compile
[INFO] | | \- (com.google.guava:guava:jar:20.0:compile - version managed from 15.0; omitted for duplicate)
[INFO] | +- org.reflections:reflections:jar:0.9.11:compile
[INFO] | | \- (com.google.guava:guava:jar:20.0:compile - version managed from 15.0; omitted for duplicate)
[INFO] | +- io.grpc:grpc-netty-shaded:jar:1.30.0:compile
[INFO] | | \- io.grpc:grpc-core:jar:1.30.0:compile
[INFO] | | \- (com.google.guava:guava:jar:20.0:runtime - version managed from 28.2-android; omitted for duplicate)
[INFO] | +- io.grpc:grpc-protobuf:jar:1.30.0:compile
[INFO] | | +- io.grpc:grpc-api:jar:1.30.0:compile
[INFO] | | | \- (com.google.guava:guava:jar:20.0:runtime - version managed from 28.2-android; omitted for duplicate)
[INFO] | | +- io.grpc:grpc-protobuf-lite:jar:1.30.0:compile
[INFO] | | | \- (com.google.guava:guava:jar:20.0:runtime - version managed from 28.2-android; omitted for duplicate)
[INFO] | | \- (com.google.guava:guava:jar:20.0:runtime - version managed from 28.2-android; omitted for duplicate)
[INFO] | \- io.grpc:grpc-stub:jar:1.30.0:compile
[INFO] | \- (com.google.guava:guava:jar:20.0:compile - version managed from 28.2-android; omitted for duplicate)
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.7.0:provided
[INFO] | +- org.apache.hadoop:hadoop-common:jar:2.7.0:provided (scope managed from compile)
[INFO] | | +- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] | | +- org.apache.hadoop:hadoop-auth:jar:2.7.0:provided
[INFO] | | | \- org.apache.curator:curator-framework:jar:2.7.1:provided
[INFO] | | | \- (com.google.guava:guava:jar:20.0:provided - version managed from 16.0.1; omitted for duplicate)
[INFO] | | +- org.apache.curator:curator-client:jar:2.7.1:provided
[INFO] | | | \- (com.google.guava:guava:jar:20.0:provided - version managed from 16.0.1; omitted for duplicate)
[INFO] | | \- org.apache.curator:curator-recipes:jar:2.7.1:provided
[INFO] | | \- (com.google.guava:guava:jar:20.0:provided - version managed from 16.0.1; omitted for duplicate)
[INFO] | +- org.apache.hadoop:hadoop-hdfs:jar:2.7.0:provided
[INFO] | | \- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.0:provided
[INFO] | | \- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.0:provided
[INFO] | | +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.0:provided
[INFO] | | | \- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] | | \- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.0:provided
[INFO] | | \- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.0:provided
[INFO] | | \- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.0:provided
[INFO] | \- org.apache.hadoop:hadoop-yarn-common:jar:2.7.0:provided
[INFO] | \- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] +- org.apache.pinot:pinot-common:jar:0.7.0-SNAPSHOT:provided
[INFO] | \- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] \- org.apache.pinot:pinot-spi:jar:0.7.0-SNAPSHOT:provided (scope not updated to compile)
[INFO] \- (com.google.guava:guava:jar:20.0:provided - version managed from 11.0.2; omitted for duplicate)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.074 s
[INFO] Finished at: 2021-04-08T09:12:35-04:00
[INFO] ------------------------------------------------------------------------
daniellavoie@Daniels-MacBook-Pro pinot-hadoop % mvn test
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.16
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 16
[INFO] os.detected.classifier: osx-x86_64
[INFO]
[INFO] -------------------< org.apache.pinot:pinot-hadoop >--------------------
[INFO] Building Pinot Hadoop 0.7.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- buildnumber-maven-plugin:1.3:create (default) @ pinot-hadoop ---
[INFO] Executing: /bin/sh -c cd /Users/daniellavoie/projects/startree/startree-pinot/git/incubator-pinot/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop && git rev-parse --verify HEAD
[INFO] Working directory: /Users/daniellavoie/projects/startree/startree-pinot/git/incubator-pinot/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop
[INFO] Storing buildNumber: 4b0db5a3cfdfe69551cef235f9d8c25ffe31b254 at timestamp: 1617887578451
[INFO] Storing buildScmBranch: master
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-dependency-convergence) @ pinot-hadoop ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ pinot-hadoop ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.7.201606060606:prepare-agent (default) @ pinot-hadoop ---
[INFO] argLine set to -javaagent:/Users/daniellavoie/.m2/repository/org/jacoco/org.jacoco.agent/0.7.7.201606060606/org.jacoco.agent-0.7.7.201606060606-runtime.jar=destfile=/Users/daniellavoie/projects/startree/startree-pinot/git/incubator-pinot/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/target/jacoco.exec -Xms4g -Xmx4g
[INFO]
[INFO] --- maven-remote-resources-plugin:1.6.0:process (process-resource-bundles) @ pinot-hadoop ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- maven-remote-resources-plugin:1.6.0:process (default) @ pinot-hadoop ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Preparing remote bundle org.apache:apache-incubator-disclaimer-resource-bundle:1.1
[INFO] Copying 4 resources from 2 bundles.
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ pinot-hadoop ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/daniellavoie/projects/startree/startree-pinot/git/incubator-pinot/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/src/main/resources
[INFO] Copying 4 resources
[INFO] Copying 4 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ pinot-hadoop ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ pinot-hadoop ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 4 resources
[INFO] Copying 4 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ pinot-hadoop ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ pinot-hadoop ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.pinot.hadoop.io.PinotOutputFormatTest
ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/daniellavoie/.m2/repository/org/apache/pinot/pinot-common/0.7.0-SNAPSHOT/pinot-common-0.7.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/daniellavoie/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.11.2/log4j-slf4j-impl-2.11.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
[ERROR] Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 1.299 s <<< FAILURE! - in org.apache.pinot.hadoop.io.PinotOutputFormatTest
[ERROR] org.apache.pinot.hadoop.io.PinotOutputFormatTest.testPinotOutputFormat Time elapsed: 0.975 s <<< FAILURE!
java.lang.NoSuchMethodError: com.google.common.cache.CacheBuilder.ticker(Lcom/google/common/base/Ticker;)Lcom/google/common/cache/CacheBuilder;
at org.apache.hadoop.security.Groups.<init>(Groups.java:99)
at org.apache.hadoop.security.Groups.<init>(Groups.java:74)
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:303)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:790)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:760)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:633)
at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:72)
at org.apache.hadoop.mapreduce.Job.<init>(Job.java:142)
at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:185)
at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:165)
at org.apache.pinot.hadoop.io.PinotOutputFormatTest.testPinotOutputFormat(PinotOutputFormatTest.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:661)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:869)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1193)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:126)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.privateRun(TestRunner.java:744)
at org.testng.TestRunner.run(TestRunner.java:602)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
at org.testng.SuiteRunner.run(SuiteRunner.java:289)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1226)
at org.testng.TestNG.runSuites(TestNG.java:1144)
at org.testng.TestNG.run(TestNG.java:1115)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:136)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:145)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
[ERROR] org.apache.pinot.hadoop.io.PinotOutputFormatTest.tearDown Time elapsed: 0.975 s <<< FAILURE!
java.io.FileNotFoundException: File does not exist: /var/folders/mx/slpjv4nd3713gnsddc6yyb_c0000gn/T/PinotOutputFormatTest
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2275)
at org.apache.pinot.hadoop.io.PinotOutputFormatTest.tearDown(PinotOutputFormatTest.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:523)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:224)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:146)
at org.testng.internal.TestMethodWorker.invokeAfterClassMethods(TestMethodWorker.java:212)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
at org.testng.TestRunner.privateRun(TestRunner.java:744)
at org.testng.TestRunner.run(TestRunner.java:602)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
at org.testng.SuiteRunner.run(SuiteRunner.java:289)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1226)
at org.testng.TestNG.runSuites(TestNG.java:1144)
at org.testng.TestNG.run(TestNG.java:1115)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:136)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:145)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
[INFO] Running org.apache.pinot.hadoop.io.DeleteExtraPushedSegmentsTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/daniellavoie/.m2/repository/org/apache/pinot/pinot-common/0.7.0-SNAPSHOT/pinot-common-0.7.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/daniellavoie/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.11.2/log4j-slf4j-impl-2.11.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.817 s - in org.apache.pinot.hadoop.io.DeleteExtraPushedSegmentsTest
[INFO] org.apache.pinot.hadoop.io.DeleteExtraPushedSegmentsTest.checkDelete Time elapsed: 0.477 s
[INFO] org.apache.pinot.hadoop.io.DeleteExtraPushedSegmentsTest.checkDeleteWithDoubleDigitSequenceIds Time elapsed: 0 s
[INFO] org.apache.pinot.hadoop.io.DeleteExtraPushedSegmentsTest.checkDeleteWithRefresh Time elapsed: 0.001 s
[INFO] org.apache.pinot.hadoop.io.DeleteExtraPushedSegmentsTest.checkDeleteWithoutSequenceIds Time elapsed: 0.001 s
[INFO] Running org.apache.pinot.hadoop.job.mappers.SegmentCreationTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.412 s - in org.apache.pinot.hadoop.job.mappers.SegmentCreationTest
[INFO] org.apache.pinot.hadoop.job.mappers.SegmentCreationTest.testBootstrapOutputPath Time elapsed: 0.042 s
[INFO] org.apache.pinot.hadoop.job.mappers.SegmentCreationTest.testBootstrapOutputPath1 Time elapsed: 0.04 s
[INFO] org.apache.pinot.hadoop.job.mappers.SegmentCreationTest.testBootstrapOutputPathHdfs Time elapsed: 0.001 s
[INFO] org.apache.pinot.hadoop.job.mappers.SegmentCreationTest.testBootstrapOutputPathS3 Time elapsed: 0.001 s
[INFO] Running org.apache.pinot.hadoop.data.IngestionSchemaValidatorTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/daniellavoie/.m2/repository/org/apache/pinot/pinot-common/0.7.0-SNAPSHOT/pinot-common-0.7.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/daniellavoie/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.11.2/log4j-slf4j-impl-2.11.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.528 s - in org.apache.pinot.hadoop.data.IngestionSchemaValidatorTest
[INFO] org.apache.pinot.hadoop.data.IngestionSchemaValidatorTest.testAvroIngestionSchemaValidatorForSingleValueColumns Time elapsed: 1.192 s
[INFO] org.apache.pinot.hadoop.data.IngestionSchemaValidatorTest.testAvroIngestionValidatorForMultiValueColumns Time elapsed: 0.003 s
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] PinotOutputFormatTest.tearDown:60 » FileNotFound File does not exist: /var/fol...
[ERROR] PinotOutputFormatTest.testPinotOutputFormat:66 » NoSuchMethod com.google.commo...
[INFO]
[ERROR] Tests run: 12, Failures: 2, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.959 s
[INFO] Finished at: 2021-04-08T09:13:09-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project pinot-hadoop: There are test failures.
[ERROR]
[ERROR] Please refer to /Users/daniellavoie/projects/startree/startree-pinot/git/incubator-pinot/pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] daniellavoie commented on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
daniellavoie commented on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815832546
Honestly, I'm clueless, hadoop-client 2.7.0 uses method signatures that matches guava 20
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815524841
I cannot reproduce this locally. Based on the error message, seems the failure is caused guava library version conflict. Can you please check if there are multiple versions of guava
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] daniellavoie commented on issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
daniellavoie commented on issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757#issuecomment-815906675
Fixed it by adding the guava `cache` package to the `pinot-common` shade relocation configuration. PR coming through.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang closed issue #6757: Test failure on Pinot Hadoop
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang closed issue #6757:
URL: https://github.com/apache/incubator-pinot/issues/6757
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org