You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Ismaël Mejía (JIRA)" <ji...@apache.org> on 2019/07/04 06:14:00 UTC
[jira] [Assigned] (BEAM-2295) Hadoop IO connectors require
additional repositories
[ https://issues.apache.org/jira/browse/BEAM-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ismaël Mejía reassigned BEAM-2295:
----------------------------------
Assignee: Jean-Baptiste Onofré (was: Jean-Baptiste Onofré)
> Hadoop IO connectors require additional repositories
> ----------------------------------------------------
>
> Key: BEAM-2295
> URL: https://issues.apache.org/jira/browse/BEAM-2295
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Pei He
> Assignee: Jean-Baptiste Onofré
> Priority: Major
> Labels: triaged
> Fix For: Not applicable
>
>
> Several dependencies are not in https://repo.maven.apache.org/maven2.
> 404 when trying to visit https://repo.maven.apache.org/maven2/cascading/
> My workaround is to add conjars.org repository to my local profile.
> <repository>
> <id>clojars</id>
> <url>https://clojars.org/repo/</url>
> <releases>
> <enabled>true</enabled>
> </releases>
> <snapshots>
> <enabled>true</enabled>
> </snapshots>
> </repository>
> The question is how to make the io connector modules have the additional repo by default? I can see this issue will come up when more io connectors are added, and when they need additional repos.
> [INFO] ------------------------------------------------------------------------
> [INFO] Building Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests 2.0.0
> [INFO] ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ beam-sdks-java-io-hadoop-jdk1.8-tests ---
> [INFO] Deleting /Users/peihe/Downloads/apache-beam-2.0.0/sdks/java/io/hadoop/jdk1.8-tests (includes = [**/*.pyc, **/*.egg-info/, **/sdks/python/LICENSE, **/sdks/python/NOTICE, **/sdks/python/README.md], excludes = [])
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce) @ beam-sdks-java-io-hadoop-jdk1.8-tests ---
> Downloading: https://repo.maven.apache.org/maven2/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
> [WARNING] Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in central (https://repo.maven.apache.org/maven2)
> Try downloading the file manually from the project website.
> Then, install it using the command:
> mvn install:install-file -DgroupId=org.pentaho -DartifactId=pentaho-aggdesigner-algorithm -Dversion=5.1.5-jhyde -Dpackaging=jar -Dfile=/path/to/file
> Alternatively, if you host your own repository you can deploy the file there:
> mvn deploy:deploy-file -DgroupId=org.pentaho -DartifactId=pentaho-aggdesigner-algorithm -Dversion=5.1.5-jhyde -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
> Path to dependency:
> 1) org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.0.0
> 2) org.elasticsearch:elasticsearch-hadoop:jar:5.0.0
> 3) org.apache.hive:hive-service:jar:1.2.1
> 4) org.apache.hive:hive-exec:jar:1.2.1
> 5) org.apache.calcite:calcite-core:jar:1.2.0-incubating
> 6) org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
> org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
> from the specified remote repositories:
> apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
> central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)
> Downloading: https://repo.maven.apache.org/maven2/cascading/cascading-hadoop/2.6.3/cascading-hadoop-2.6.3.jar
> [WARNING] Could not find artifact cascading:cascading-hadoop:jar:2.6.3 in central (https://repo.maven.apache.org/maven2)
> Try downloading the file manually from the project website.
> Then, install it using the command:
> mvn install:install-file -DgroupId=cascading -DartifactId=cascading-hadoop -Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file
> Alternatively, if you host your own repository you can deploy the file there:
> mvn deploy:deploy-file -DgroupId=cascading -DartifactId=cascading-hadoop -Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
> Path to dependency:
> 1) org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.0.0
> 2) org.elasticsearch:elasticsearch-hadoop:jar:5.0.0
> 3) cascading:cascading-hadoop:jar:2.6.3
> cascading:cascading-hadoop:jar:2.6.3
> from the specified remote repositories:
> apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
> central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)
> Downloading: https://repo.maven.apache.org/maven2/cascading/cascading-local/2.6.3/cascading-local-2.6.3.jar
> [WARNING] Could not find artifact cascading:cascading-local:jar:2.6.3 in central (https://repo.maven.apache.org/maven2)
> Try downloading the file manually from the project website.
> Then, install it using the command:
> mvn install:install-file -DgroupId=cascading -DartifactId=cascading-local -Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file
> Alternatively, if you host your own repository you can deploy the file there:
> mvn deploy:deploy-file -DgroupId=cascading -DartifactId=cascading-local -Dversion=2.6.3 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
> Path to dependency:
> 1) org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.0.0
> 2) org.elasticsearch:elasticsearch-hadoop:jar:5.0.0
> 3) cascading:cascading-local:jar:2.6.3
> cascading:cascading-local:jar:2.6.3
> from the specified remote repositories:
> apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
> central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)