You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Patrice DUROUX (Jira)" <ji...@apache.org> on 2021/10/27 12:50:00 UTC
[jira] [Commented] (SPARK-37130) why
spark-X.X.X-bin-without-hadoop.tgz does not provide spark-hive_X.jar (and
spark-hive-thriftserver_X.jar)
[ https://issues.apache.org/jira/browse/SPARK-37130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434848#comment-17434848 ]
Patrice DUROUX commented on SPARK-37130:
----------------------------------------
ps: a diff output
{{$ diff spark-without.lst spark-with.lst }}
{{667a668}}
{{> /jars/activation-1.1.1.jar}}
{{671a673}}
{{> /jars/antlr-runtime-3.5.2.jar}}
{{684a687}}
{{> /jars/bonecp-0.8.0.RELEASE.jar}}
{{689a693}}
{{> /jars/commons-cli-1.2.jar}}
{{694a699}}
{{> /jars/commons-dbcp-1.4.jar}}
{{695a701}}
{{> /jars/commons-lang-2.6.jar}}
{{696a703}}
{{> /jars/commons-logging-1.1.3.jar}}
{{698a706}}
{{> /jars/commons-pool-1.5.4.jar}}
{{701a710,717}}
{{> /jars/curator-client-2.13.0.jar}}
{{> /jars/curator-framework-2.13.0.jar}}
{{> /jars/curator-recipes-2.13.0.jar}}
{{> /jars/datanucleus-api-jdo-4.2.4.jar}}
{{> /jars/datanucleus-core-4.1.17.jar}}
{{> /jars/datanucleus-rdbms-4.1.19.jar}}
{{> /jars/derby-10.14.2.0.jar}}
{{> /jars/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar}}
{{704c720,739}}
{{< /jars/gson-2.8.6.jar}}
{{---}}
{{> /jars/gson-2.2.4.jar}}
{{> /jars/guava-14.0.1.jar}}
{{> /jars/hadoop-client-api-3.3.1.jar}}
{{> /jars/hadoop-client-runtime-3.3.1.jar}}
{{> /jars/hadoop-shaded-guava-1.1.1.jar}}
{{> /jars/hadoop-yarn-server-web-proxy-3.3.1.jar}}
{{> /jars/HikariCP-2.5.1.jar}}
{{> /jars/hive-beeline-2.3.9.jar}}
{{> /jars/hive-cli-2.3.9.jar}}
{{> /jars/hive-common-2.3.9.jar}}
{{> /jars/hive-exec-2.3.9-core.jar}}
{{> /jars/hive-jdbc-2.3.9.jar}}
{{> /jars/hive-llap-common-2.3.9.jar}}
{{> /jars/hive-metastore-2.3.9.jar}}
{{> /jars/hive-serde-2.3.9.jar}}
{{> /jars/hive-service-rpc-3.1.2.jar}}
{{> /jars/hive-shims-0.23-2.3.9.jar}}
{{> /jars/hive-shims-2.3.9.jar}}
{{> /jars/hive-shims-common-2.3.9.jar}}
{{> /jars/hive-shims-scheduler-2.3.9.jar}}
{{705a741}}
{{> /jars/hive-vector-code-gen-2.3.9.jar}}
{{708a745,747}}
{{> /jars/htrace-core4-4.1.0-incubating.jar}}
{{> /jars/httpclient-4.5.13.jar}}
{{> /jars/httpcore-4.4.14.jar}}
{{712a752}}
{{> /jars/jackson-core-asl-1.9.13.jar}}
{{715a756}}
{{> /jars/jackson-mapper-asl-1.9.13.jar}}
{{724a766,767}}
{{> /jars/javax.jdo-3.2.0-m3.jar}}
{{> /jars/javolution-5.5.1.jar}}
{{727a771}}
{{> /jars/jdo-api-3.0.1.jar}}
{{734a779,783}}
{{> /jars/jline-2.14.6.jar}}
{{> /jars/joda-time-2.10.10.jar}}
{{> /jars/jodd-core-3.5.2.jar}}
{{> /jars/jpam-1.1.jar}}
{{> /jars/json-1.8.jar}}
{{739a789}}
{{> /jars/jta-1.1.jar}}
{{765a816,818}}
{{> /jars/libfb303-0.9.3.jar}}
{{> /jars/libthrift-0.12.0.jar}}
{{> /jars/log4j-1.2.17.jar}}
{{792a846}}
{{> /jars/protobuf-java-2.5.0.jar}}
{{804a859,860}}
{{> /jars/slf4j-api-1.7.30.jar}}
{{> /jars/slf4j-log4j12-1.7.30.jar}}
{{809a866,867}}
{{> /jars/spark-hive_2.12-3.2.0.jar}}
{{> /jars/spark-hive-thriftserver_2.12-3.2.0.jar}}
{{829a888,889}}
{{> /jars/ST4-4.0.4.jar}}
{{> /jars/stax-api-1.0.1.jar}}
{{830a891}}
{{> /jars/super-csv-2.2.0.jar}}
{{832a894}}
{{> /jars/transaction-api-1.1.jar}}
{{833a896}}
{{> /jars/velocity-1.5.jar}}
{{836a900,901}}
{{> /jars/zookeeper-3.6.2.jar}}
{{> /jars/zookeeper-jute-3.6.2.jar}}
{{919a985}}
{{> /python/dist/}}
{{1015a1082,1087}}
{{> /python/pyspark.egg-info/}}
{{> /python/pyspark.egg-info/dependency_links.txt}}
{{> /python/pyspark.egg-info/PKG-INFO}}
{{> /python/pyspark.egg-info/requires.txt}}
{{> /python/pyspark.egg-info/SOURCES.txt}}
{{> /python/pyspark.egg-info/top_level.txt}}
{{1269a1342,1346}}
{{> /python/pyspark/__pycache__/}}
{{> /python/pyspark/__pycache__/install.cpython-38.pyc}}
{{> /python/pyspark/python/}}
{{> /python/pyspark/python/pyspark/}}
{{> /python/pyspark/python/pyspark/shell.py}}
{{1497a1575,1579}}
{{> /R/lib/SparkR/doc/}}
{{> /R/lib/SparkR/doc/index.html}}
{{> /R/lib/SparkR/doc/sparkr-vignettes.html}}
{{> /R/lib/SparkR/doc/sparkr-vignettes.R}}
{{> /R/lib/SparkR/doc/sparkr-vignettes.Rmd}}
{{1514a1597}}
{{> /R/lib/SparkR/Meta/vignette.rds}}
> why spark-X.X.X-bin-without-hadoop.tgz does not provide spark-hive_X.jar (and spark-hive-thriftserver_X.jar)
> ------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-37130
> URL: https://issues.apache.org/jira/browse/SPARK-37130
> Project: Spark
> Issue Type: Improvement
> Components: Deploy
> Affects Versions: 3.1.2, 3.2.0
> Reporter: Patrice DUROUX
> Priority: Minor
>
> Hi,
> As my deployment is having its own Hadoop(+Hive) installed, I have tried to install Spark using its bundle without Hadoop. I suspect that some jars are missing that are present in the corresponding spark-X.X.X-bin-hadoop3.2.tgz.
> After comparing their contents both spark-hive_2.12-X.X.X.jar and spark-hive-thriftserver_2.12-X.X.X.jar are not in the spark-X.X.X-bin-without---hadoop.tgz. And I don't know if some others should also be there.
> Thanks,
> Patrice
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org