You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by KaiXinXiaoLei <gi...@git.apache.org> on 2014/10/29 04:47:32 UTC
[GitHub] spark pull request: delete jetty 6.1.26 form spark package
GitHub user KaiXinXiaoLei opened a pull request:
https://github.com/apache/spark/pull/2989
delete jetty 6.1.26 form spark package
In spark code, jetty version is 8.1.14.v20131031 (see ./pom.xml). But when building, the jetty-6.1.26 and jetty-util-6.1.26 is packaged in spark-assembly*.jar. So the version 8.1.14.v20131031 and 6.1.26 of jetty are existed in spark-assembly*.jar.
When achieving SslSocketConnector using jetty in Spark web, this is a problem. Because jetty v6 dose not support sslContextFactory(), should make own SslSocketConnector . See: http://stackoverflow.com/questions/26382540/how-to-disable-the-sslv3-protocol-in-jetty-to-prevent-poodle-attack
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/KaiXinXiaoLei/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2989.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2989
----
commit 7624ebdcc227e3469cac30e2c20457434736078c
Author: KaiXinXiaoLei <hu...@huawei.com>
Date: 2014-10-29T02:16:26Z
change pom.xml
delete jetty 6.1.26
commit dff8f456a02a17fcc8bb9c56c1cc1fd521ad6f20
Author: KaiXinXiaoLei <hu...@huawei.com>
Date: 2014-10-29T02:18:15Z
change pom.xml
when packaging, delete org/mortbay/**
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Delete jetty 6.1.26 from spark package
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2989
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: delete jetty 6.1.26 form spark package
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2989#issuecomment-60870455
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Delete jetty 6.1.26 from spark package
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2989#issuecomment-63355824
The older jetty code (jetty 6.X) is under different package names and I think it is used by Hadoop and other transitive dependencies. So I think we need to close this issue because it is not safe to do this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Delete jetty 6.1.26 from spark package
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/2989#issuecomment-60907459
Other libraries may be using old Jetty. It is not clearly safe to do this. Can you identify first what is bringing in Jetty 6?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [GitHub] spark pull request: Delete jetty 6.1.26 from spark package
Posted by Sean Owen <so...@cloudera.com>.
Yeah, Hadoop uses it. I am not sure you can just take it out?
On Oct 30, 2014 4:11 AM, "KaiXinXiaoLei" <gi...@git.apache.org> wrote:
> Github user KaiXinXiaoLei commented on the pull request:
>
> https://github.com/apache/spark/pull/2989#issuecomment-61040188
>
> Using the maven-dependency-plugin to build spark, I get the dependency
> tree for spark. I find the jetty 6 is introduced by hdfs ,yarn, flume and
> hbase. From the info of dependency tree, here I just give the information
> about Jetty 6.
> The Jetty 6 is brought by hdfs when building spark-core:
> [INFO] +- org.apache.hadoop:hadoop-client:jar:2.4.1:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs:jar:2.4.1:compile
> [INFO] | | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
>
> The Jetty 6 is brought by yarn when building spark-yarn:
> [INFO] +-
> org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.4.1:compile
> [INFO] | +-
> org.apache.hadoop:hadoop-yarn-server-common:jar:2.4.1:compile
> [INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
> [INFO] | \- org.mortbay.jetty:jetty:jar:6.1.26:compile
> [INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.4.1:compile
> [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
>
> The Jetty 6 is brought by flume when building spark-streaming-flume:
> [INFO] +-
> org.apache.spark:spark-streaming-flume-sink_2.10:jar:1.2.0-SNAPSHOT:compile
> [INFO] | \- org.apache.flume:flume-ng-core:jar:1.4.0:compile
> [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
>
> The Jetty 6 is brought by hbase when building spark-examples:
> [INFO] +- org.apache.hbase:hbase:jar:0.94.6:compile
> [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
> [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
> For additional commands, e-mail: reviews-help@spark.apache.org
>
>
[GitHub] spark pull request: Delete jetty 6.1.26 from spark package
Posted by KaiXinXiaoLei <gi...@git.apache.org>.
Github user KaiXinXiaoLei commented on the pull request:
https://github.com/apache/spark/pull/2989#issuecomment-61040188
Using the maven-dependency-plugin to build spark, I get the dependency tree for spark. I find the jetty 6 is introduced by hdfs ,yarn, flume and hbase. From the info of dependency tree, here I just give the information about Jetty 6.
The Jetty 6 is brought by hdfs when building spark-core:
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.4.1:compile
[INFO] | +- org.apache.hadoop:hadoop-hdfs:jar:2.4.1:compile
[INFO] | | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
The Jetty 6 is brought by yarn when building spark-yarn:
[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.4.1:compile
[INFO] | +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.4.1:compile
[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | \- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.4.1:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
The Jetty 6 is brought by flume when building spark-streaming-flume:
[INFO] +- org.apache.spark:spark-streaming-flume-sink_2.10:jar:1.2.0-SNAPSHOT:compile
[INFO] | \- org.apache.flume:flume-ng-core:jar:1.4.0:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
The Jetty 6 is brought by hbase when building spark-examples:
[INFO] +- org.apache.hbase:hbase:jar:0.94.6:compile
[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Delete jetty 6.1.26 from spark package
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2989#issuecomment-63356418
Yeah, unless there is proof that the Hadoop clients won't need any of those classes, it's safer to leave them there. If you really care about pruning dependencies, you can use `-Phadoop-provided` in your build - although the dependencies might still come from the hadoop jars you'll need to add to your classpath.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org