You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by bo...@apache.org on 2015/10/02 22:21:57 UTC
[1/3] storm git commit: Shaded everything that was not already shaded,
and cleaned up build so the dependency-reduced pom.xml is used correctly.
Repository: storm
Updated Branches:
refs/heads/0.10.x-branch 42a3d9bd1 -> a2af30d20
Shaded everything that was not already shaded, and cleaned up build so the dependency-reduced pom.xml is used correctly.
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/503174ec
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/503174ec
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/503174ec
Branch: refs/heads/0.10.x-branch
Commit: 503174ec88cc43e69cc952d736d201deeb0ff185
Parents: 42a3d9b
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Authored: Fri Sep 11 11:56:34 2015 -0500
Committer: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Committed: Fri Oct 2 14:55:12 2015 -0500
----------------------------------------------------------------------
DEVELOPER.md | 53 ++--
dev-tools/travis/travis-install.sh | 4 +-
dev-tools/travis/travis-script.sh | 5 +-
external/storm-eventhubs/pom.xml | 2 +-
external/storm-kafka/pom.xml | 5 +
pom.xml | 114 +++++++--
storm-core/pom.xml | 257 ++++++++++++--------
storm-core/src/clj/backtype/storm/log.clj | 2 +-
storm-dist/binary/src/main/assembly/binary.xml | 30 ---
9 files changed, 279 insertions(+), 193 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/DEVELOPER.md
----------------------------------------------------------------------
diff --git a/DEVELOPER.md b/DEVELOPER.md
index b3c4c9c..295e461 100644
--- a/DEVELOPER.md
+++ b/DEVELOPER.md
@@ -195,6 +195,7 @@ To pull in a merge request you should generally follow the command line instruct
3. Merge the pull request into your local test branch.
$ git pull <remote_repo_url> <remote_branch>
+ You can use `./dev-tools/storm-merge.py <pull-number>` to produce the above command most of the time.
4. Assuming that the pull request merges without any conflicts:
Update the top-level `CHANGELOG.md`, and add in the JIRA ticket number (example: `STORM-1234`) and ticket
@@ -225,49 +226,47 @@ To pull in a merge request you should generally follow the command line instruct
# Build the code and run the tests
-The following commands must be run from the top-level directory.
-
- # Build the code and run the tests (requires nodejs, python and ruby installed)
- # `mvn clean package` will fail because storm-core requires storm-maven-plugin.
- # This plugin should be installed before compiling storm-core.
- $ mvn clean install
+## Prerequisites
+Firt of all you need to make sure you are using maven 3.2.5 or below. There is a bug in later versions of maven as linked to from https://issues.apache.org/jira/browse/MSHADE-206 that
+cause shaded dependencies to not be packaged correctly. Also please be aware that because we are shading dependencies mvn dependency:tree will not always show the dependencies correctly.
- # Build the code and run the tests, with specifying default test timeout (in millisecond)
- $ export STORM_TEST_TIMEOUT_MS=10000
- $ mvn clean install
+In order to build `storm` you need `python`, `ruby` and `nodejs`. In order to avoid an overful page we don't provide platform/OS specific installation instructions for those here. Please refer to you platform's/OS' documentation for support.
- # Build the code but skip the tests
- $ mvn clean install -DskipTests=true
+The `ruby` package manager `rvm` and `nodejs` package manager `nvm` are for convenience and are used in the tests which run on [travis](https://travis-ci.org/apache/storm). They can be installed using `curl -L https://get.rvm.io | bash -s stable --autolibs=enabled && source ~/.profile` (see the [rvm installation instructions](https://github.com/rvm/rvm) for details) and `wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.26.1/install.sh | bash && source ~/.bashrc` (see the [nvm installation instructions](https://github.com/creationix/nvm) for details).
-In case you modified `storm.thrift`, you have to regenerate thrift code as java and python code before compiling whole project.
+With `rvm` and `nvm` installed you can run
```sh
-cd storm-core/src
-sh genthrift.sh
+rvm use 2.1.5 --install
+nvm install 0.12.2
+nvm use 0.12.2
```
-You can also run tests selectively via the Clojure REPL. The following example runs the tests in
-[auth_test.clj](storm-core/test/clj/backtype/storm/security/auth/auth_test.clj), which has the namespace
-`backtype.storm.security.auth.auth-test`.
+in order to get started as fast as possible. Users can still install a specific version of `ruby` and/or `node` manually.
+
+## Building
+
+The following commands must be run from the top-level directory.
-First, start the REPL from within the relevant sub-project (here: `storm-core`):
+`mvn clean install`
- $ cd storm-core/
- $ mvn clojure:repl
+If you wish to skip the unit tests you can do this by adding `-DskipTests` to the command line.
-Now we run the tests in `auth_test.clj` in the REPL:
+In case you modified `storm.thrift`, you have to regenerate thrift code as java and python code before compiling whole project.
-```clojure
-;; You can use both absolute as well as relative paths to the .clj file.
-(load-file "test/clj/backtype/storm/security/auth/auth_test.clj")
-(ns backtype.storm.security.auth.auth-test)
-(run-tests)
+```sh
+cd storm-core/src
+sh genthrift.sh
```
+You can also run tests selectively with `-Dtest=<test_name>`. This works for both clojure and junit tests.
+
+
> Tip: IDEs such as IntelliJ IDEA support a built-in Clojure REPL, which you can also use to run tests selectively.
> Sometimes you may find that tests pass/fail depending on which REPL you use, which -- although frustrating --
> can be helpful to narrow down errors.
+Unfortunately you might experience failures in clojure tests which are wrapped in the `maven-clojure-plugin` and thus doesn't provide too much useful output at first sight - you might end up with a maven test failure with an error message as unhelpful as `Clojure failed.`. In this case it's recommended to look into `target/test-reports` of the failed project to see what actual tests have failed or scroll through the maven output looking for obvious issues like missing binaries.
<a name="packaging"></a>
@@ -277,7 +276,7 @@ You can create a _distribution_ (like what you can download from Apache) as foll
do not use the Maven release plugin because creating an official release is the task of our release manager.
# First, build the code.
- $ mvn clean install # you may skip tests with `-DskipTests=true` to save time
+ $ mvn clean install # you may skip tests with `-DskipTests=true` to save time
# Create the binary distribution.
$ cd storm-dist/binary && mvn package
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/dev-tools/travis/travis-install.sh
----------------------------------------------------------------------
diff --git a/dev-tools/travis/travis-install.sh b/dev-tools/travis/travis-install.sh
index 2812240..8f6ce21 100755
--- a/dev-tools/travis/travis-install.sh
+++ b/dev-tools/travis/travis-install.sh
@@ -15,10 +15,12 @@ PYTHON_VERSION_TO_FILE=`python -V > /tmp/python_version 2>&1`
PYTHON_VERSION=`cat /tmp/python_version`
RUBY_VERSION=`ruby -v`
NODEJS_VERSION=`node -v`
+MVN_VERSION=`mvn -v`
echo "Python version : $PYTHON_VERSION"
echo "Ruby version : $RUBY_VERSION"
echo "NodeJs version : $NODEJS_VERSION"
+echo "mvn version : $MVN_VERSION"
STORM_SRC_ROOT_DIR=$1
@@ -26,7 +28,7 @@ TRAVIS_SCRIPT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
cd ${STORM_SRC_ROOT_DIR}
-python ${TRAVIS_SCRIPT_DIR}/save-logs.py "install.txt" mvn clean install -DskipTests -Pnative
+python ${TRAVIS_SCRIPT_DIR}/save-logs.py "install.txt" mvn clean install -DskipTests -Pnative --batch-mode
BUILD_RET_VAL=$?
if [[ "$BUILD_RET_VAL" != "0" ]];
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/dev-tools/travis/travis-script.sh
----------------------------------------------------------------------
diff --git a/dev-tools/travis/travis-script.sh b/dev-tools/travis/travis-script.sh
index 77b9fb6..dd1a337 100755
--- a/dev-tools/travis/travis-script.sh
+++ b/dev-tools/travis/travis-script.sh
@@ -15,11 +15,12 @@ PYTHON_VERSION_TO_FILE=`python -V > /tmp/python_version 2>&1`
PYTHON_VERSION=`cat /tmp/python_version`
RUBY_VERSION=`ruby -v`
NODEJS_VERSION=`node -v`
+MVN_VERSION=`mvn -v`
echo "Python version : $PYTHON_VERSION"
echo "Ruby version : $RUBY_VERSION"
echo "NodeJs version : $NODEJS_VERSION"
-
+echo "mvn version : $MVN_VERSION"
STORM_SRC_ROOT_DIR=$1
@@ -31,7 +32,7 @@ cd ${STORM_SRC_ROOT_DIR}
export STORM_TEST_TIMEOUT_MS=100000
# We now lean on Travis CI's implicit behavior, ```mvn clean install -DskipTests``` before running script
-mvn test -fae -Pnative
+mvn --batch-mode install -fae -Pnative
BUILD_RET_VAL=$?
for dir in `find . -type d -and -wholename \*/target/\*-reports`;
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/external/storm-eventhubs/pom.xml
----------------------------------------------------------------------
diff --git a/external/storm-eventhubs/pom.xml b/external/storm-eventhubs/pom.xml
index beada09..b036fec 100755
--- a/external/storm-eventhubs/pom.xml
+++ b/external/storm-eventhubs/pom.xml
@@ -110,4 +110,4 @@
<scope>test</scope>
</dependency>
</dependencies>
-</project>
\ No newline at end of file
+</project>
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/external/storm-kafka/pom.xml
----------------------------------------------------------------------
diff --git a/external/storm-kafka/pom.xml b/external/storm-kafka/pom.xml
index fa948cd..2805f2a 100644
--- a/external/storm-kafka/pom.xml
+++ b/external/storm-kafka/pom.xml
@@ -57,6 +57,11 @@
</build>
<dependencies>
<dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ <scope>compile</scope>
+ </dependency>
+ <dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.9.0</version>
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 8c5929f..71c39d7 100644
--- a/pom.xml
+++ b/pom.xml
@@ -154,24 +154,6 @@
<maven>3.0.0</maven>
</prerequisites>
- <modules>
- <module>storm-buildtools/maven-shade-clojure-transformer</module>
- <module>storm-buildtools/storm-maven-plugins</module>
- <module>storm-multilang/javascript</module>
- <module>storm-multilang/python</module>
- <module>storm-multilang/ruby</module>
- <module>storm-core</module>
- <module>examples/storm-starter</module>
- <module>external/storm-kafka</module>
- <module>external/storm-hdfs</module>
- <module>external/storm-hbase</module>
- <module>external/storm-hive</module>
- <module>external/storm-jdbc</module>
- <module>external/storm-redis</module>
- <module>external/storm-eventhubs</module>
- <module>external/flux</module>
- </modules>
-
<scm>
<connection>scm:git:https://git-wip-us.apache.org/repos/asf/storm.git</connection>
<developerConnection>scm:git:https://git-wip-us.apache.org/repos/asf/storm.git</developerConnection>
@@ -195,10 +177,13 @@
<commons-io.version>2.4</commons-io.version>
<commons-lang.version>2.5</commons-lang.version>
<commons-exec.version>1.1</commons-exec.version>
+ <commons-fileupload.version>1.2.1</commons-fileupload.version>
+ <commons-codec.version>1.6</commons-codec.version>
<clj-time.version>0.8.0</clj-time.version>
<curator.version>2.5.0</curator.version>
<json-simple.version>1.1</json-simple.version>
<ring.version>1.3.0</ring.version>
+ <ring-json.version>0.3.1</ring-json.version>
<jetty.version>7.6.13.v20130916</jetty.version>
<clojure.tools.logging.version>0.2.3</clojure.tools.logging.version>
<clojure.math.numeric-tower.version>0.0.1</clojure.math.numeric-tower.version>
@@ -212,6 +197,7 @@
<netty.version>3.9.0.Final</netty.version>
<log4j-over-slf4j.version>1.6.6</log4j-over-slf4j.version>
<log4j.version>2.1</log4j.version>
+ <slf4j.version>1.7.7</slf4j.version>
<clojure.tools.nrepl.version>0.2.3</clojure.tools.nrepl.version>
<clojure-complete.version>0.2.3</clojure-complete.version>
<mockito.version>1.9.5</mockito.version>
@@ -222,8 +208,32 @@
<hive.version>0.14.0</hive.version>
<hadoop.version>2.6.0</hadoop.version>
<kryo.version>2.21</kryo.version>
+ <servlet.version>2.5</servlet.version>
+ <joda-time.version>2.3</joda-time.version>
+ <jackson.version>2.3.1</jackson.version>
+ <thrift.version>0.9.2</thrift.version>
+ <junit.version>4.11</junit.version>
</properties>
+ <modules>
+ <module>storm-multilang/javascript</module>
+ <module>storm-multilang/python</module>
+ <module>storm-multilang/ruby</module>
+ <module>storm-buildtools/maven-shade-clojure-transformer</module>
+ <module>storm-buildtools/storm-maven-plugins</module>
+ <module>storm-core</module>
+ <module>external/storm-kafka</module>
+ <module>external/storm-hdfs</module>
+ <module>external/storm-hbase</module>
+ <module>external/storm-hive</module>
+ <module>external/storm-jdbc</module>
+ <module>external/storm-redis</module>
+ <module>external/storm-eventhubs</module>
+ <module>external/flux</module>
+ <module>examples/storm-starter</module>
+ </modules>
+
+
<profiles>
<profile>
<id>sign</id>
@@ -378,6 +388,41 @@
</exclusions>
</dependency>
<dependency>
+ <groupId>javax.servlet</groupId>
+ <artifactId>servlet-api</artifactId>
+ <version>${servlet.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>joda-time</groupId>
+ <artifactId>joda-time</artifactId>
+ <version>${joda-time.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-core</artifactId>
+ <version>${jackson.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.dataformat</groupId>
+ <artifactId>jackson-dataformat-smile</artifactId>
+ <version>${jackson.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>commons-fileupload</groupId>
+ <artifactId>commons-fileupload</artifactId>
+ <version>${commons-fileupload.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>commons-codec</groupId>
+ <artifactId>commons-codec</artifactId>
+ <version>${commons-codec.version}</version>
+ </dependency>
+ <dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
<version>${curator.version}</version>
@@ -418,6 +463,11 @@
<version>${ring.version}</version>
</dependency>
<dependency>
+ <groupId>ring</groupId>
+ <artifactId>ring-json</artifactId>
+ <version>${ring-json.version}</version>
+ </dependency>
+ <dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlet</artifactId>
<version>${jetty.version}</version>
@@ -428,6 +478,17 @@
<version>${jetty.version}</version>
</dependency>
<dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-server</artifactId>
+ <version>${jetty.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-util</artifactId>
+ <version>${jetty.version}</version>
+ </dependency>
+
+ <dependency>
<groupId>org.clojure</groupId>
<artifactId>tools.logging</artifactId>
<version>${clojure.tools.logging.version}</version>
@@ -552,16 +613,15 @@
<dependency>
<groupId>org.apache.thrift</groupId>
<artifactId>libthrift</artifactId>
- <version>0.9.2</version>
+ <version>${thrift.version}</version>
<scope>compile</scope>
</dependency>
- <!-- used by examples/storm-starter -->
- <dependency>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- <version>4.11</version>
- <scope>test</scope>
- </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>test</scope>
+ </dependency>
</dependencies>
</dependencyManagement>
@@ -655,7 +715,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
- <version>2.2</version>
+ <version>2.4.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/storm-core/pom.xml
----------------------------------------------------------------------
diff --git a/storm-core/pom.xml b/storm-core/pom.xml
index bd5ba2d..4e367c9 100644
--- a/storm-core/pom.xml
+++ b/storm-core/pom.xml
@@ -39,6 +39,12 @@
<dependency>
<groupId>com.esotericsoftware.kryo</groupId>
<artifactId>kryo</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>org.objenesis</groupId>
+ <artifactId>objenesis</artifactId>
+ </exclusion>
+ </exclusions>
</dependency>
<!--clojure-->
<dependency>
@@ -74,7 +80,6 @@
<dependency>
<groupId>ring</groupId>
<artifactId>ring-json</artifactId>
- <version>0.3.1</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
@@ -141,7 +146,6 @@
<dependency>
<groupId>org.apache.thrift</groupId>
<artifactId>libthrift</artifactId>
- <version>0.9.2</version>
<scope>compile</scope>
<exclusions>
<exclusion>
@@ -154,8 +158,6 @@
</exclusion>
</exclusions>
</dependency>
-
-
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
@@ -176,8 +178,6 @@
<artifactId>json-simple</artifactId>
<scope>compile</scope>
</dependency>
-
-
<dependency>
<groupId>com.twitter</groupId>
<artifactId>carbonite</artifactId>
@@ -291,10 +291,49 @@
</exclusions>
</dependency>
<dependency>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- <version>4.11</version>
- <scope>test</scope>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.zookeeper</groupId>
+ <artifactId>zookeeper</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>javax.servlet</groupId>
+ <artifactId>servlet-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>joda-time</groupId>
+ <artifactId>joda-time</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-server</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.eclipse.jetty</groupId>
+ <artifactId>jetty-util</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.dataformat</groupId>
+ <artifactId>jackson-dataformat-smile</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>commons-fileupload</groupId>
+ <artifactId>commons-fileupload</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>commons-codec</groupId>
+ <artifactId>commons-codec</artifactId>
</dependency>
</dependencies>
<build>
@@ -408,23 +447,22 @@
<minimizeJar>false</minimizeJar>
<artifactSet>
<includes>
+ <include>ns-tracker:ns-tracker</include>
+ <include>hiccup:hiccup</include>
+ <include>ring:*</include>
+ <include>compojure:compojure</include>
+ <include>clj-time:clj-time</include>
<include>org.apache.thrift:*</include>
<include>io.netty:netty</include>
<include>com.google.guava:guava</include>
<include>org.apache.httpcomponents:http*</include>
<include>org.apache.zookeeper:zookeeper</include>
<include>org.apache.curator:*</include>
- <include>org.apache.httpcomponents:http*</include>
- <include>org.apache.zookeeper:zookeeper</include>
- <include>org.apache.curator:*</include>
<include>com.twitter:carbonite</include>
<include>com.twitter:chill-java</include>
- <include>org.objenesis:objenesis</include>
<include>org.tukaani:xz</include>
<include>org.yaml:snakeyaml</include>
<include>org.jgrapht:jgrapht-core</include>
- <include>commons-httpclient:commons-httpclient</include>
- <include>org.apache.commons:commons-compress</include>
<include>org.apache.commons:commons-exec</include>
<include>commons-io:commons-io</include>
<include>commons-codec:commons-codec</include>
@@ -433,158 +471,205 @@
<include>com.googlecode.json-simple:json-simple</include>
<include>org.clojure:math.numeric-tower</include>
<include>org.clojure:tools.cli</include>
+ <include>org.clojure:tools.logging</include>
<include>org.clojure:tools.macro</include>
+ <include>org.clojure:java.jmx</include>
<include>joda-time:joda-time</include>
<include>org.eclipse.jetty:*</include>
+ <include>com.fasterxml.jackson.core:*</include>
+ <include>com.fasterxml.jackson.dataformat:*</include>
+ <include>clout:clout</include>
+ <include>org.clojure:tools.namespace</include>
+ <include>cheshire:cheshire</include>
+ <include>org.clojure:core.incubator</include>
</includes>
</artifactSet>
-
<relocations>
<relocation>
- <pattern>org.apache.thrift</pattern>
- <shadedPattern>org.apache.thrift7</shadedPattern>
+ <pattern>cheshire</pattern>
+ <shadedPattern>org.apache.storm.shade.cheshire</shadedPattern>
</relocation>
<relocation>
- <pattern>org.jboss.netty</pattern>
- <shadedPattern>org.apache.storm.netty</shadedPattern>
+ <pattern>clojure.tools.logging</pattern>
+ <shadedPattern>org.apache.storm.shade.clojure.tools.logging</shadedPattern>
</relocation>
<relocation>
- <pattern>com.google.common</pattern>
- <shadedPattern>org.apache.storm.guava</shadedPattern>
+ <pattern>clojure.core.incubator</pattern>
+ <shadedPattern>org.apache.storm.shade.clojure.core.incubator</shadedPattern>
</relocation>
<relocation>
- <pattern>com.google.thirdparty</pattern>
- <shadedPattern>org.apache.storm.guava.thirdparty</shadedPattern>
+ <pattern>clojure.tools.namespace</pattern>
+ <shadedPattern>org.apache.storm.shade.clojure.tools.namespace</shadedPattern>
</relocation>
<relocation>
- <pattern>org.apache.http</pattern>
- <shadedPattern>org.apache.storm.http</shadedPattern>
+ <pattern>clout</pattern>
+ <shadedPattern>org.apache.storm.shade.clout</shadedPattern>
</relocation>
<relocation>
- <pattern>org.apache.zookeeper</pattern>
- <shadedPattern>org.apache.storm.zookeeper</shadedPattern>
+ <pattern>compojure</pattern>
+ <shadedPattern>org.apache.storm.shade.compojure</shadedPattern>
</relocation>
<relocation>
- <pattern>org.apache.curator</pattern>
- <shadedPattern>org.apache.storm.curator</shadedPattern>
+ <pattern>ns_tracker</pattern>
+ <shadedPattern>org.apache.storm.shade.ns_tracker</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>ns-tracker</pattern>
+ <shadedPattern>org.apache.storm.shade.ns-tracker</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>hiccup</pattern>
+ <shadedPattern>org.apache.storm.shade.hiccup</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>ring</pattern>
+ <shadedPattern>org.apache.storm.shade.ring</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>clj_time</pattern>
+ <shadedPattern>org.apache.storm.shade.clj_time</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>clj-time</pattern>
+ <shadedPattern>org.apache.storm.shade.clj-time</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.fasterxml</pattern>
+ <shadedPattern>org.apache.storm.shade.com.fasterxml</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>org.apache.thrift</pattern>
+ <!-- This pattern is inconsistent for backwards compatibility purposes. -->
+ <shadedPattern>org.apache.thrift7</shadedPattern>
</relocation>
<relocation>
<pattern>org.jboss.netty</pattern>
- <shadedPattern>org.apache.storm.netty</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.jboss.netty</shadedPattern>
</relocation>
<relocation>
<pattern>com.google.common</pattern>
- <shadedPattern>org.apache.storm.guava</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.com.google.common</shadedPattern>
</relocation>
<relocation>
<pattern>com.google.thirdparty</pattern>
- <shadedPattern>org.apache.storm.guava.thirdparty</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.com.google.thirdparty</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.http</pattern>
- <shadedPattern>org.apache.storm.http</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.http</shadedPattern>
</relocation>
<relocation>
- <pattern>org.apache.zookeeper</pattern>
- <shadedPattern>org.apache.storm.zookeeper</shadedPattern>
+ <pattern>org.apache.curator</pattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.curator</shadedPattern>
</relocation>
<relocation>
- <pattern>org.apache.jute</pattern>
- <shadedPattern>org.apache.storm.jute</shadedPattern>
+ <pattern>org.apache.zookeeper</pattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.zookeeper</shadedPattern>
</relocation>
<relocation>
- <pattern>org.apache.curator</pattern>
- <shadedPattern>org.apache.storm.curator</shadedPattern>
+ <pattern>org.apache.jute</pattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.jute</shadedPattern>
</relocation>
<relocation>
<pattern>carbonite</pattern>
- <shadedPattern>org.apache.storm.carbonite</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.carbonite</shadedPattern>
</relocation>
<relocation>
<pattern>com.twitter.chill</pattern>
- <shadedPattern>org.apache.storm.chill</shadedPattern>
- </relocation>
- <relocation>
- <pattern>org.objenesis</pattern>
- <shadedPattern>org.apache.storm.objenesis</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.com.twitter.chill</shadedPattern>
</relocation>
<relocation>
<pattern>org.tukaani.xz</pattern>
- <shadedPattern>org.apache.storm.xz</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.tukaani.xz</shadedPattern>
</relocation>
<relocation>
<pattern>org.yaml.snakeyaml</pattern>
- <shadedPattern>org.apache.storm.snakeyaml</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.yaml.snakeyaml</shadedPattern>
</relocation>
<relocation>
<pattern>org.jgrapht</pattern>
- <shadedPattern>org.apache.storm.jgrapht</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.jgrapht</shadedPattern>
</relocation>
<relocation>
<pattern>org.fusesource</pattern>
- <shadedPattern>org.apache.storm.fusesource</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.fusesource</shadedPattern>
</relocation>
<relocation>
<pattern>com.metamx.http.client</pattern>
- <shadedPattern>org.apache.storm.metamx.http.client</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.com.metamx.http.client</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.commons.io</pattern>
- <shadedPattern>org.apache.storm.commons.io</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.commons.io</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.commons.codec</pattern>
- <shadedPattern>org.apache.storm.commons.codec</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.commons.codec</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.commons.fileupload</pattern>
- <shadedPattern>org.apache.storm.commons.fileupload</shadedPattern>
- </relocation>
- <relocation>
- <pattern>org.apache.commons.compress</pattern>
- <shadedPattern>org.apache.storm.commons.compress</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.commons.fileupload</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.commons.exec</pattern>
- <shadedPattern>org.apache.storm.commons.exec</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.commons.exec</shadedPattern>
</relocation>
<relocation>
<pattern>org.apache.commons.lang</pattern>
- <shadedPattern>org.apache.storm.commons.lang</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.apache.commons.lang</shadedPattern>
</relocation>
<relocation>
<pattern>org.json.simple</pattern>
- <shadedPattern>org.apache.storm.json.simple</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.json.simple</shadedPattern>
</relocation>
<relocation>
<pattern>clojure.math</pattern>
- <shadedPattern>org.apache.storm.clojure.math</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.clojure.math</shadedPattern>
</relocation>
<relocation>
<pattern>clojure.tools.cli</pattern>
- <shadedPattern>org.apache.storm.clojure.tools.cli</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.clojure.tools.cli</shadedPattern>
</relocation>
<relocation>
<pattern>cljs.tools.cli</pattern>
- <shadedPattern>org.apache.storm.cljs.tools.cli</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.cljs.tools.cli</shadedPattern>
</relocation>
<relocation>
<pattern>clojure.tools.macro</pattern>
- <shadedPattern>org.apache.storm.clojure.tools.macro</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.clojure.tools.macro</shadedPattern>
</relocation>
<relocation>
<pattern>org.joda.time</pattern>
- <shadedPattern>org.apache.storm.joda.time</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.joda.time</shadedPattern>
</relocation>
<relocation>
<pattern>org.eclipse.jetty</pattern>
- <shadedPattern>org.apache.storm.jetty</shadedPattern>
+ <shadedPattern>org.apache.storm.shade.org.eclipse.jetty</shadedPattern>
</relocation>
</relocations>
<transformers>
<transformer implementation="org.apache.storm.maven.shade.clojure.ClojureTransformer" />
</transformers>
<filters>
+ <!-- Several of these filters remove the .clj files from the shaded dependencies, even though only .clj files are in these jars.
+ The reason for this is a bit complex, but intentional. During the build process all of the dependency .clj files are
+ compiled down into .class files, and included in storm-core.jar. The regular shade transformer handles these in
+ the majority of cases correctly. However, the Clojure-Transformer does not shade everything correctly all the
+ time. Instead of spending a lot of time to get the Clojure-Transformer to parse Clojure correctly we opted to remove
+ the .clj files from the uber jar. -->
+ <filter><artifact>org.clojure:core.incubator</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>cheshire:cheshire</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>org.clojure:tools.logging</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>org.clojure:tools.namespace</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>org.clojure:math.numeric-tower</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>org.clojure:tools.macro</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>org.clojure:tools.cli</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>ns-tracker:ns-tracker</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>clout:clout</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>hiccup:hiccup</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>clj-time:clj-time</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>ring:*</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
+ <filter><artifact>compojure:compojure</artifact><excludes><exclude>**/*.clj</exclude></excludes></filter>
<filter>
<artifact>org.apache.thrift:*</artifact>
<excludes>
@@ -600,48 +685,12 @@
</excludes>
</filter>
<filter>
- <artifact>commons-httpclient:commons-httpclient</artifact>
- <excludes>
- <exclude>META-INF/LICENSE.txt</exclude>
- <exclude>META-INF/NOTICE.txt</exclude>
- <exclude>META-INF/README.txt</exclude>
- </excludes>
- </filter>
- <filter>
- <artifact>org.apache.zookeeper:zookeeper</artifact>
- <excludes>
- <exclude>LICENSE.txt</exclude>
- </excludes>
- </filter>
- <filter>
- <artifact>commons-httpclient:commons-httpclient</artifact>
- <excludes>
- <exclude>META-INF/LICENSE.txt</exclude>
- <exclude>META-INF/NOTICE.txt</exclude>
- <exclude>META-INF/README.txt</exclude>
- </excludes>
- </filter>
- <filter>
<artifact>org.apache.zookeeper:zookeeper</artifact>
<excludes>
<exclude>LICENSE.txt</exclude>
</excludes>
</filter>
<filter>
- <artifact>org.objenesis:objenesis</artifact>
- <excludes>
- <exclude>META-INF/LICENSE.txt</exclude>
- <exclude>META-INF/NOTICE.txt</exclude>
- </excludes>
- </filter>
- <filter>
- <artifact>org.apache.commons:commons-compress</artifact>
- <excludes>
- <exclude>META-INF/LICENSE.txt</exclude>
- <exclude>META-INF/NOTICE.txt</exclude>
- </excludes>
- </filter>
- <filter>
<artifact>org.apache.commons:commons-exec</artifact>
<excludes>
<exclude>META-INF/LICENSE.txt</exclude>
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/storm-core/src/clj/backtype/storm/log.clj
----------------------------------------------------------------------
diff --git a/storm-core/src/clj/backtype/storm/log.clj b/storm-core/src/clj/backtype/storm/log.clj
index 0fcf822..55b44fe 100644
--- a/storm-core/src/clj/backtype/storm/log.clj
+++ b/storm-core/src/clj/backtype/storm/log.clj
@@ -15,7 +15,7 @@
;; limitations under the License.
(ns backtype.storm.log
- (:require [clojure.tools [logging :as log]]))
+ (:require [clojure.tools.logging :as log]))
(defmacro log-message
[& args]
http://git-wip-us.apache.org/repos/asf/storm/blob/503174ec/storm-dist/binary/src/main/assembly/binary.xml
----------------------------------------------------------------------
diff --git a/storm-dist/binary/src/main/assembly/binary.xml b/storm-dist/binary/src/main/assembly/binary.xml
index a63a6f1..d35ab96 100644
--- a/storm-dist/binary/src/main/assembly/binary.xml
+++ b/storm-dist/binary/src/main/assembly/binary.xml
@@ -30,36 +30,6 @@
<useProjectArtifact>false</useProjectArtifact>
<outputDirectory>lib</outputDirectory>
<unpack>false</unpack>
- <excludes>
- <exclude>org.apache.thrift:*</exclude>
- <exclude>io.netty:netty</exclude>
- <exclude>com.google.guava:guava</exclude>
- <exclude>org.apache.httpcomponents:http*</exclude>
- <exclude>org.apache.zookeeper:zookeeper</exclude>
- <exclude>org.apache.curator:*</exclude>
- <exclude>org.apache.httpcomponents:http*</exclude>
- <exclude>org.apache.zookeeper:zookeeper</exclude>
- <exclude>org.apache.curator:*</exclude>
- <exclude>com.twitter:carbonite</exclude>
- <exclude>com.twitter:chill-java</exclude>
- <exclude>org.objenesis:objenesis</exclude>
- <exclude>org.tukaani:xz</exclude>
- <exclude>org.yaml:snakeyaml</exclude>
- <exclude>org.jgrapht:jgrapht-core</exclude>
- <exclude>commons-httpclient:commons-httpclient</exclude>
- <exclude>org.apache.commons:commons-compress</exclude>
- <exclude>org.apache.commons:commons-exec</exclude>
- <exclude>commons-io:commons-io</exclude>
- <exclude>commons-codec:commons-codec</exclude>
- <exclude>commons-fileupload:commons-fileupload</exclude>
- <exclude>commons-lang:commons-lang</exclude>
- <exclude>com.googlecode.json-simple:json-simple</exclude>
- <exclude>org.clojure:math.numeric-tower</exclude>
- <exclude>org.clojure:tools.cli</exclude>
- <exclude>org.clojure:tools.macro</exclude>
- <exclude>joda-time:joda-time</exclude>
- <exclude>org.eclipse.jetty:*</exclude>
- </excludes>
</dependencySet>
</dependencySets>
[3/3] storm git commit: Updated Changelog
Posted by bo...@apache.org.
Updated Changelog
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/a2af30d2
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/a2af30d2
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/a2af30d2
Branch: refs/heads/0.10.x-branch
Commit: a2af30d204b402202ef4fcd2ece60771ad7b07ca
Parents: 53a2c06
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Authored: Fri Oct 2 15:21:05 2015 -0500
Committer: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Committed: Fri Oct 2 15:21:05 2015 -0500
----------------------------------------------------------------------
CHANGELOG.md | 4 ++++
1 file changed, 4 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/a2af30d2/CHANGELOG.md
----------------------------------------------------------------------
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f3c68a3..19f8e0b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,8 @@
## 0.10.0
+ * STORM-1012: Shaded everything that was not already shaded
+ * STORM-967: Shaded everything that was not already shaded
+ * STORM-922: Shaded everything that was not already shaded
+ * STORM-1042: Shaded everything that was not already shaded
* STORM-1026: Adding external classpath elements does not work
* STORM-1055: storm-jdbc README needs fixes and context
* STORM-1044: Setting dop to zero does not raise an error
[2/3] storm git commit: Merge branch 'STORM-1042-0.10.x' of
https://github.com/revans2/incubator-storm into STORM-1042
Posted by bo...@apache.org.
Merge branch 'STORM-1042-0.10.x' of https://github.com/revans2/incubator-storm into STORM-1042
STORM-1012 STORM-967 STORM-922 STORM-1042 Shaded everything that was not already shaded
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/53a2c065
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/53a2c065
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/53a2c065
Branch: refs/heads/0.10.x-branch
Commit: 53a2c06574c942985e13fbfcdef7f56c8f0de4c7
Parents: 42a3d9b 503174e
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Authored: Fri Oct 2 15:20:05 2015 -0500
Committer: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Committed: Fri Oct 2 15:20:05 2015 -0500
----------------------------------------------------------------------
DEVELOPER.md | 53 ++--
dev-tools/travis/travis-install.sh | 4 +-
dev-tools/travis/travis-script.sh | 5 +-
external/storm-eventhubs/pom.xml | 2 +-
external/storm-kafka/pom.xml | 5 +
pom.xml | 114 +++++++--
storm-core/pom.xml | 257 ++++++++++++--------
storm-core/src/clj/backtype/storm/log.clj | 2 +-
storm-dist/binary/src/main/assembly/binary.xml | 30 ---
9 files changed, 279 insertions(+), 193 deletions(-)
----------------------------------------------------------------------