You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Wensheng Deng <we...@yahoo.com.INVALID> on 2021/05/19 17:41:46 UTC
unresolved dependency:
graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found
Hi experts:
I tried the example as shown on this page, and it is not working for me:https://spark-packages.org/package/graphframes/graphframes
Please advise how to proceed. I also tried to unzip the zip file, ran 'sbt assembly', and got an error of 'sbt-spark-package;0.2.6: not found'. Is there a solution to this?
$ spark-shell --packages graphframes:graphframes:0.8.1-spark2.4-s_2.11Ivy Default Cache set to: /Users/wsd57/.ivy2/cacheThe jars for the packages stored in: /Users/wsd57/.ivy2/jars:: loading settings :: url = jar:file:/Users/wsd57/spark-2.4.0-bin-hadoop2.6/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xmlgraphframes#graphframes added as a dependency:: resolving dependencies :: org.apache.spark#spark-submit-parent-eb9df503-f763-49b3-83e9-5afef8b0311d;1.0 confs: [default]:: resolution report :: resolve 1429ms :: artifacts dl 0ms :: modules in use: --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 1 | 0 | 0 | 0 || 0 | 0 | ---------------------------------------------------------------------
:: problems summary :::::: WARNINGS module not found: graphframes#graphframes;0.8.1-spark2.4-s_2.11
==== local-m2-cache: tried
file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
==== local-ivy-cache: tried
/Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/ivys/ivy.xml
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
/Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/jars/graphframes.jar
==== central: tried
https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
::::::::::::::::::::::::::::::::::::::::::::::
:: UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found
::::::::::::::::::::::::::::::::::::::::::::::
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILSException in thread "main" java.lang.RuntimeException: [unresolved dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found] at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1306) at org.apache.spark.deploy.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:54) at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:315) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:143) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Thank you!
RegardsW. Deng
Re: unresolved dependency:
graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found
Posted by Wensheng Deng <we...@yahoo.com.INVALID>.
Thanks Sean. Your are right! Yes it works when replacing the bintray repo with repos.spark-packages.org.
On Wednesday, May 19, 2021, 02:03:14 PM EDT, Sean Owen <sr...@gmail.com> wrote:
I think it's because the bintray repo has gone away. Did you see the recent email about the new repo for these packages?
On Wed, May 19, 2021 at 12:42 PM Wensheng Deng <we...@yahoo.com.invalid> wrote:
Hi experts:
I tried the example as shown on this page, and it is not working for me:https://spark-packages.org/package/graphframes/graphframes
Please advise how to proceed. I also tried to unzip the zip file, ran 'sbt assembly', and got an error of 'sbt-spark-package;0.2.6: not found'. Is there a solution to this?
$ spark-shell --packages graphframes:graphframes:0.8.1-spark2.4-s_2.11Ivy Default Cache set to: /Users/wsd57/.ivy2/cacheThe jars for the packages stored in: /Users/wsd57/.ivy2/jars:: loading settings :: url = jar:file:/Users/wsd57/spark-2.4.0-bin-hadoop2.6/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xmlgraphframes#graphframes added as a dependency:: resolving dependencies :: org.apache.spark#spark-submit-parent-eb9df503-f763-49b3-83e9-5afef8b0311d;1.0 confs: [default]:: resolution report :: resolve 1429ms :: artifacts dl 0ms :: modules in use: --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 1 | 0 | 0 | 0 || 0 | 0 | ---------------------------------------------------------------------
:: problems summary :::::: WARNINGS module not found: graphframes#graphframes;0.8.1-spark2.4-s_2.11
==== local-m2-cache: tried
file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
==== local-ivy-cache: tried
/Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/ivys/ivy.xml
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
/Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/jars/graphframes.jar
==== central: tried
https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
-- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
::::::::::::::::::::::::::::::::::::::::::::::
:: UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found
::::::::::::::::::::::::::::::::::::::::::::::
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILSException in thread "main" java.lang.RuntimeException: [unresolved dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found] at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1306) at org.apache.spark.deploy.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:54) at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:315) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:143) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Thank you!
RegardsW. Deng
Re: unresolved dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11:
not found
Posted by Sean Owen <sr...@gmail.com>.
I think it's because the bintray repo has gone away. Did you see the recent
email about the new repo for these packages?
On Wed, May 19, 2021 at 12:42 PM Wensheng Deng <we...@yahoo.com.invalid>
wrote:
> Hi experts:
>
> I tried the example as shown on this page, and it is not working for me:
> https://spark-packages.org/package/graphframes/graphframes
>
> Please advise how to proceed. I also tried to unzip the zip file, ran 'sbt
> assembly', and got an error of 'sbt-spark-package;0.2.6: not found'. Is
> there a solution to this?
>
> $ spark-shell --packages graphframes:graphframes:0.8.1-spark2.4-s_2.11
> Ivy Default Cache set to: /Users/wsd57/.ivy2/cache
> The jars for the packages stored in: /Users/wsd57/.ivy2/jars
> :: loading settings :: url =
> jar:file:/Users/wsd57/spark-2.4.0-bin-hadoop2.6/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
> graphframes#graphframes added as a dependency
> :: resolving dependencies ::
> org.apache.spark#spark-submit-parent-eb9df503-f763-49b3-83e9-5afef8b0311d;1.0
> confs: [default]
> :: resolution report :: resolve 1429ms :: artifacts dl 0ms
> :: modules in use:
> ---------------------------------------------------------------------
> | | modules || artifacts |
> | conf | number| search|dwnlded|evicted|| number|dwnlded|
> ---------------------------------------------------------------------
> | default | 1 | 0 | 0 | 0 || 0 | 0 |
> ---------------------------------------------------------------------
>
> :: problems summary ::
> :::: WARNINGS
> module not found: graphframes#graphframes;0.8.1-spark2.4-s_2.11
>
> ==== local-m2-cache: tried
>
>
> file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
>
> -- artifact
> graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
>
>
> file:/Users/wsd57/.m2/repository/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
>
> ==== local-ivy-cache: tried
>
>
> /Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/ivys/ivy.xml
>
> -- artifact graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
>
>
> /Users/wsd57/.ivy2/local/graphframes/graphframes/0.8.1-spark2.4-s_2.11/jars/graphframes.jar
>
> ==== central: tried
>
>
> https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
>
> -- artifact
> graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
>
>
> https://repo1.maven.org/maven2/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
>
> ==== spark-packages: tried
>
>
> http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.pom
>
> -- artifact
> graphframes#graphframes;0.8.1-spark2.4-s_2.11!graphframes.jar:
>
>
> http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.8.1-spark2.4-s_2.11/graphframes-0.8.1-spark2.4-s_2.11.jar
>
> ::::::::::::::::::::::::::::::::::::::::::::::
>
> :: UNRESOLVED DEPENDENCIES ::
>
> ::::::::::::::::::::::::::::::::::::::::::::::
>
> :: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found
>
> ::::::::::::::::::::::::::::::::::::::::::::::
>
>
>
> :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> Exception in thread "main" java.lang.RuntimeException: [unresolved
> dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found]
> at
> org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1306)
> at
> org.apache.spark.deploy.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:54)
> at
> org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:315)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:143)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
>
> Thank you!
>
> Regards
> W. Deng
>
>