You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Chesnay Schepler (Jira)" <ji...@apache.org> on 2019/12/06 15:18:00 UTC
[jira] [Closed] (FLINK-13945) Instructions for building
flink-shaded against vendor repository don't work
[ https://issues.apache.org/jira/browse/FLINK-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chesnay Schepler closed FLINK-13945.
------------------------------------
Fix Version/s: 1.9.2
1.10.0
Resolution: Fixed
master: a61f26124285447de89d4a63baab4c39e2423cb0
1.9: e4ad3b035da5958a4beb0c0d6d8d0124a5c54b7c
> Instructions for building flink-shaded against vendor repository don't work
> ---------------------------------------------------------------------------
>
> Key: FLINK-13945
> URL: https://issues.apache.org/jira/browse/FLINK-13945
> Project: Flink
> Issue Type: Bug
> Components: BuildSystem / Shaded
> Affects Versions: 1.9.0
> Reporter: Elise Ramé
> Assignee: Chesnay Schepler
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0, 1.9.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> According to [documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.9/flinkDev/building.html#custom--vendor-specific-versions], to build Flink against a vendor specific Hadoop version it is necessary to build flink-shaded against this version first :
> {code:bash}
> mvn clean install -DskipTests -Pvendor-repos -Dhadoop.version=<hadoop_version>
> {code}
> vendor-repos profile has to be activated to include Hadoop vendors repositories.
> But Maven cannot find expected Hadoop dependencies and returns an error because vendor-repos profile isn't defined in flink-shaded.
> Example using flink-shaded 8.0 and HDP 2.6.5 Hadoop version :
> {code:bash}
> mvn clean install -DskipTests -Pvendor-repos -Dhadoop.version=2.7.3.2.6.5.0-292
> {code}
> {code:bash}
> [INFO] ---------------< org.apache.flink:flink-shaded-hadoop-2 >---------------
> [INFO] Building flink-shaded-hadoop-2 2.7.3.2.6.5.0-292-8.0 [10/11]
> [INFO] --------------------------------[ jar ]---------------------------------
> [WARNING] The POM for org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-292 is missing, no dependency information available
> [WARNING] The POM for org.apache.hadoop:hadoop-hdfs:jar:2.7.3.2.6.5.0-292 is missing, no dependency information available
> [WARNING] The POM for org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3.2.6.5.0-292 is missing, no dependency information available
> [WARNING] The POM for org.apache.hadoop:hadoop-yarn-client:jar:2.7.3.2.6.5.0-292 is missing, no dependency information available
> [WARNING] The POM for org.apache.hadoop:hadoop-yarn-common:jar:2.7.3.2.6.5.0-292 is missing, no dependency information available
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] flink-shaded 8.0 ................................... SUCCESS [ 2.122 s]
> [INFO] flink-shaded-force-shading 8.0 ..................... SUCCESS [ 0.607 s]
> [INFO] flink-shaded-asm-7 7.1-8.0 ......................... SUCCESS [ 0.667 s]
> [INFO] flink-shaded-guava-18 18.0-8.0 ..................... SUCCESS [ 1.452 s]
> [INFO] flink-shaded-netty-4 4.1.39.Final-8.0 .............. SUCCESS [ 4.597 s]
> [INFO] flink-shaded-netty-tcnative-dynamic 2.0.25.Final-8.0 SUCCESS [ 0.620 s]
> [INFO] flink-shaded-jackson-parent 2.9.8-8.0 .............. SUCCESS [ 0.018 s]
> [INFO] flink-shaded-jackson-2 2.9.8-8.0 ................... SUCCESS [ 0.914 s]
> [INFO] flink-shaded-jackson-module-jsonSchema-2 2.9.8-8.0 . SUCCESS [ 0.627 s]
> [INFO] flink-shaded-hadoop-2 2.7.3.2.6.5.0-292-8.0 ........ FAILURE [ 0.047 s]
> [INFO] flink-shaded-hadoop-2-uber 2.7.3.2.6.5.0-292-8.0 ... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 11.947 s
> [INFO] Finished at: 2019-09-03T16:52:59+02:00
> [INFO] ------------------------------------------------------------------------
> [WARNING] The requested profile "vendor-repos" could not be activated because it does not exist.
> [ERROR] Failed to execute goal on project flink-shaded-hadoop-2: Could not resolve dependencies for project org.apache.flink:flink-shaded-hadoop-2:jar:2.7.3.2.6.5.0-292-8.0: The following artifacts could not be resolved: org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-292, org.apache.hadoop:hadoop-hdfs:jar:2.7.3.2.6.5.0-292, org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3.2.6.5.0-292, org.apache.hadoop:hadoop-yarn-client:jar:2.7.3.2.6.5.0-292, org.apache.hadoop:hadoop-yarn-common:jar:2.7.3.2.6.5.0-292: Failure to find org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-292 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the command
> [ERROR] mvn <goals> -rf :flink-shaded-hadoop-2
> {code}
> vendor-repos profile exists in Flink pom.xml file : [https://github.com/apache/flink/blob/3079d11913f153ec40c75afb5356fd3be1a1e550/pom.xml#L1037]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)