You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Vishal Donderia (Jira)" <ji...@apache.org> on 2020/07/13 05:32:00 UTC
[jira] [Comment Edited] (OOZIE-2973) Make sure Oozie works with
Hadoop 3
[ https://issues.apache.org/jira/browse/OOZIE-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17156489#comment-17156489 ]
Vishal Donderia edited comment on OOZIE-2973 at 7/13/20, 5:31 AM:
------------------------------------------------------------------
Are we able to compiled oozie using hadoop-3 ? I am using below command to compile oozie-5.2.0
{code:java}
./bin/mkdistro.sh -DskipTests -Phadoop-3 -Dhadoop.version=3.0.0 -X
{code}
And getting below exception
{code:java}
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:testCompile (default-testCompile) on project oozie-tools: Compilation failure
[ERROR] /Users/vishal/Documents/Softwares/BigData/oozie-5.2.0/tools/src/test/java/org/apache/oozie/tools/TestECPolicyDisabler.java:[50,71] getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.oozie.tools.TestECPolicyDisabler.MockDistributedFileSystem cannot override getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.hadoop.hdfs.DistributedFileSystem
[ERROR] return type org.apache.hadoop.hdfs.protocol.SystemErasureCodingPolicies.ReplicationPolicy is not compatible with org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
[ERROR]
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:testCompile (default-testCompile) on project oozie-tools: Compilation failure
/Users/vishal/Documents/Softwares/BigData/oozie-5.2.0/tools/src/test/java/org/apache/oozie/tools/TestECPolicyDisabler.java:[50,71] getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.oozie.tools.TestECPolicyDisabler.MockDistributedFileSystem cannot override getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.hadoop.hdfs.DistributedFileSystem
return type org.apache.hadoop.hdfs.protocol.SystemErasureCodingPolicies.ReplicationPolicy is not compatible with org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
/Users/vishal/Documents/Softwares/BigData/oozie-5.2.0/tools/src/test/java/org/apache/oozie/tools/TestECPolicyDisabler.java:[50,71] getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.oozie.tools.TestECPolicyDisabler.MockDistributedFileSystem cannot override getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.hadoop.hdfs.DistributedFileSystem
return type org.apache.hadoop.hdfs.protocol.SystemErasureCodingPolicies.ReplicationPolicy is not compatible with org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1161)
at org.apache.maven.plugin.compiler.TestCompilerMojo.execute (TestCompilerMojo.java:176)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :oozie-tools
ERROR, Oozie distro creation failed
{code}
was (Author: vishaldonderia@gmail.com):
Are we able to compiled oozie using hadoop-3. I am using below command
{code}
./bin/mkdistro.sh -DskipTests -Phadoop-3 -Dhadoop.version=3.0.0 -X
{code}
And getting below exception
{code}[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:testCompile (default-testCompile) on project oozie-tools: Compilation failure
[ERROR] /Users/vishal/Documents/Softwares/BigData/oozie-5.2.0/tools/src/test/java/org/apache/oozie/tools/TestECPolicyDisabler.java:[50,71] getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.oozie.tools.TestECPolicyDisabler.MockDistributedFileSystem cannot override getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.hadoop.hdfs.DistributedFileSystem
[ERROR] return type org.apache.hadoop.hdfs.protocol.SystemErasureCodingPolicies.ReplicationPolicy is not compatible with org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
[ERROR]
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:testCompile (default-testCompile) on project oozie-tools: Compilation failure
/Users/vishal/Documents/Softwares/BigData/oozie-5.2.0/tools/src/test/java/org/apache/oozie/tools/TestECPolicyDisabler.java:[50,71] getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.oozie.tools.TestECPolicyDisabler.MockDistributedFileSystem cannot override getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.hadoop.hdfs.DistributedFileSystem
return type org.apache.hadoop.hdfs.protocol.SystemErasureCodingPolicies.ReplicationPolicy is not compatible with org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
/Users/vishal/Documents/Softwares/BigData/oozie-5.2.0/tools/src/test/java/org/apache/oozie/tools/TestECPolicyDisabler.java:[50,71] getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.oozie.tools.TestECPolicyDisabler.MockDistributedFileSystem cannot override getErasureCodingPolicy(org.apache.hadoop.fs.Path) in org.apache.hadoop.hdfs.DistributedFileSystem
return type org.apache.hadoop.hdfs.protocol.SystemErasureCodingPolicies.ReplicationPolicy is not compatible with org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy
at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1161)
at org.apache.maven.plugin.compiler.TestCompilerMojo.execute (TestCompilerMojo.java:176)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :oozie-tools
ERROR, Oozie distro creation failed
{code}
> Make sure Oozie works with Hadoop 3
> ------------------------------------
>
> Key: OOZIE-2973
> URL: https://issues.apache.org/jira/browse/OOZIE-2973
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.3.0
> Reporter: Attila Sasvári
> Priority: Blocker
> Fix For: 5.3.0
>
> Attachments: OOZIE-2973.001.patch, OOZIE-2973.002.patch
>
>
> This JIRA is to track that Oozie works with Hadoop 3.
> - Build Oozie with latest hadoop 3 (e.g. assuming Apache Hadoop 3.0.0-alpha3 pass {{-D-Dhadoop.version=3.0.0-alpha3}} to maven) and execute tests
> - It would be nice to create an Apache jenkins job that runs Oozie tests with hadoop 3 (also with other component)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)