You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by go...@apache.org on 2016/04/13 08:13:08 UTC
[6/8] incubator-slider git commit: SLIDER-1099 Kafka app package
build should be moved to a profile (Vasanth kumar RJ via billie)
SLIDER-1099 Kafka app package build should be moved to a profile (Vasanth kumar RJ via billie)
Project: http://git-wip-us.apache.org/repos/asf/incubator-slider/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-slider/commit/9263426a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-slider/tree/9263426a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-slider/diff/9263426a
Branch: refs/heads/feature/SLIDER-906_docker_support
Commit: 9263426adca495c5ddbbf3f2765abaa75c353703
Parents: e5ebd26
Author: Billie Rinaldi <bi...@gmail.com>
Authored: Thu Apr 7 08:11:24 2016 -0700
Committer: Billie Rinaldi <bi...@gmail.com>
Committed: Thu Apr 7 08:11:24 2016 -0700
----------------------------------------------------------------------
app-packages/kafka/README.md | 27 ++--
app-packages/kafka/pom.xml | 158 +++++++++----------
.../kafka/src/assembly/koya-app-package.xml | 4 +-
3 files changed, 92 insertions(+), 97 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-slider/blob/9263426a/app-packages/kafka/README.md
----------------------------------------------------------------------
diff --git a/app-packages/kafka/README.md b/app-packages/kafka/README.md
index 0a3de5a..0341093 100644
--- a/app-packages/kafka/README.md
+++ b/app-packages/kafka/README.md
@@ -18,8 +18,6 @@
Kafka On YARN (KOYA)
====================
-[](https://gitter.im/DataTorrent/koya?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
-
### Goals
* Use capabilities of YARN for Kafka broker management
@@ -36,24 +34,23 @@ Kafka as YARN application using Slider
Checkout Slider code (https://github.com/apache/incubator-slider)
```sh
-git clone git@github.com:apache/incubator-slider.git
-git checkout -b slider-0.80.0-incubating remotes/origin/releases/slider-0.80.0-incubating
+git clone https://git-wip-us.apache.org/repos/asf/incubator-slider.git
+cd incubator-slider
+git checkout -b develop remotes/origin/develop
```
-The Slider version you checked out needs to match ${slider.version} in pom.xml
-Create symbolic link to Slider source code within the KOYA repository:
+Download Kafka binary package (http://kafka.apache.org/downloads.html), e.g.
```sh
-ln -s /path/to/repo/incubator-slider/ slider
+wget -O /tmp/kafka_2.10-0.8.2.1.tgz https://dist.apache.org/repos/dist/release/kafka/0.8.2.1/kafka_2.10-0.8.2.1.tgz
```
-Download Kafka binary package (http://kafka.apache.org/downloads.html)
+Build Kafka app package
```sh
-mvn clean install -DskipTests -Dkafka.src=path/to/kafka_2.10-0.8.1.1.tgz -Dkafka.version=kafka_2.10-0.8.1.1
+mvn clean package -DskipTests -Pkafka-app-package -pkg.src=/tmp -Dpkg.name=kafka_2.10-0.8.2.1.tgz
```
Artifacts:
- - Archive with embedded Slider: __`target/koya-with-slider.zip`__
- - Separate Slider application package: __`target/slider-kafka-app-package-0.90.0-incubating-SNAPSHOT.zip`__
+ - Slider application package: __`app-packages/kafka/target/slider-kafka-app-package-0.90.0-incubating-SNAPSHOT.zip`__
###Installation
@@ -61,14 +58,14 @@ Artifacts:
To use the archive with embedded Slider, copy it to the machine from which you launch YARN applications (Hadoop client, gateway or edge node). Extract the file and configure Slider:
-If the environment variables `HADOOP_CONF_DIR` or `JAVA_HOME` are not already defined through your Hadoop installation, you can export them in `slider-0.80.0-incubating/conf/slider-env.sh`
+If the environment variables `HADOOP_CONF_DIR` or `JAVA_HOME` are not already defined through your Hadoop installation, you can export them in `slider-0.90.0-incubating/conf/slider-env.sh`
-Example for CDH 5.4:
+Example:
```
export HADOOP_CONF_DIR=/etc/hadoop/conf
-export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+export JAVA_HOME=/usr/jdk64/jdk1.8.0_60
```
-If the registry ZooKeeper quorum was not already configured through Hadoop, modify `slider-0.80.0-incubating/conf/slider-client.xml`:
+If the registry ZooKeeper quorum was not already configured through Hadoop, modify `slider-0.90.0-incubating/conf/slider-client.xml`:
```
<property>
<name>hadoop.registry.zk.quorum</name>
http://git-wip-us.apache.org/repos/asf/incubator-slider/blob/9263426a/app-packages/kafka/pom.xml
----------------------------------------------------------------------
diff --git a/app-packages/kafka/pom.xml b/app-packages/kafka/pom.xml
index d491109..8e54455 100644
--- a/app-packages/kafka/pom.xml
+++ b/app-packages/kafka/pom.xml
@@ -28,85 +28,6 @@
<name>Slider Kafka App Package</name>
<description>Slider Kafka App Package</description>
- <properties>
- <kafka.src>${project.build.directory}/kafka_2.10-0.8.2.1.tgz</kafka.src>
- <kafka.version>kafka_2.10-0.8.2.1</kafka.version>
- </properties>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-assembly-plugin</artifactId>
- <version>${maven-assembly-plugin.version}</version>
- <executions>
- <execution>
- <id>app-package</id>
- <phase>package</phase>
- <goals>
- <goal>single</goal>
- </goals>
- <configuration>
- <descriptor>src/assembly/koya-app-package.xml</descriptor>
- <appendAssemblyId>false</appendAssemblyId>
- <finalName>${app.package.name}</finalName>
- </configuration>
- </execution>
- <execution>
- <id>package-with-slider</id>
- <phase>package</phase>
- <goals>
- <goal>single</goal>
- </goals>
- <configuration>
- <descriptor>src/assembly/koya-with-slider.xml</descriptor>
- <appendAssemblyId>false</appendAssemblyId>
- <finalName>slider-kafka-app-package-with-slider-${project.version}</finalName>
- <attach>false</attach>
- </configuration>
- </execution>
-
- </executions>
- </plugin>
- <plugin>
- <artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
- <dependencies>
- <dependency>
- <groupId>org.codehaus.groovy</groupId>
- <artifactId>groovy-eclipse-compiler</artifactId>
- <version>${groovy-eclipse-compiler.version}</version>
- </dependency>
- <dependency>
- <groupId>org.codehaus.groovy</groupId>
- <artifactId>groovy-eclipse-batch</artifactId>
- <version>${groovy-eclipse-batch.version}</version>
- </dependency>
- </dependencies>
- </plugin>
- <plugin>
- <groupId>org.codehaus.mojo</groupId>
- <artifactId>wagon-maven-plugin</artifactId>
- <version>${wagon-maven-plugin.version}</version>
- <executions>
- <execution>
- <id>download-kafka</id>
- <phase>prepare-package</phase>
- <goals>
- <goal>download-single</goal>
- </goals>
- <configuration>
- <url>https://dist.apache.org</url>
- <fromFile>repos/dist/release/kafka/0.8.2.1/kafka_2.10-0.8.2.1.tgz</fromFile>
- <toDir>${project.build.directory}</toDir>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
-
- </build>
-
<dependencies>
<dependency>
<groupId>org.apache.slider</groupId>
@@ -116,6 +37,64 @@
<profiles>
<profile>
+ <id>kafka-app-package</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-assembly-plugin</artifactId>
+ <version>${maven-assembly-plugin.version}</version>
+ <executions>
+ <execution>
+ <id>app-package</id>
+ <phase>package</phase>
+ <goals>
+ <goal>single</goal>
+ </goals>
+ <configuration>
+ <descriptor>src/assembly/koya-app-package.xml</descriptor>
+ <appendAssemblyId>false</appendAssemblyId>
+ <finalName>${app.package.name}</finalName>
+ </configuration>
+ </execution>
+ <execution>
+ <id>package-with-slider</id>
+ <phase>package</phase>
+ <goals>
+ <goal>single</goal>
+ </goals>
+ <configuration>
+ <descriptor>src/assembly/koya-with-slider.xml</descriptor>
+ <appendAssemblyId>false</appendAssemblyId>
+ <finalName>slider-kafka-app-package-with-slider-${project.version}</finalName>
+ <attach>false</attach>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <version>${maven-antrun-plugin.version}</version>
+ <executions>
+ <execution>
+ <id>copy</id>
+ <phase>validate</phase>
+ <configuration>
+ <target name="copy and rename file">
+ <copy file="${pkg.src}/${pkg.name}" tofile="${project.build.directory}/${pkg.name}" />
+ </target>
+ </configuration>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ <profile>
<id>rat</id>
<build>
<plugins>
@@ -142,5 +121,24 @@
</build>
</profile>
</profiles>
-
+ <build>
+ <plugins>
+ <plugin>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>${maven-compiler-plugin.version}</version>
+ <dependencies>
+ <dependency>
+ <groupId>org.codehaus.groovy</groupId>
+ <artifactId>groovy-eclipse-compiler</artifactId>
+ <version>${groovy-eclipse-compiler.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.codehaus.groovy</groupId>
+ <artifactId>groovy-eclipse-batch</artifactId>
+ <version>${groovy-eclipse-batch.version}</version>
+ </dependency>
+ </dependencies>
+ </plugin>
+ </plugins>
+ </build>
</project>
http://git-wip-us.apache.org/repos/asf/incubator-slider/blob/9263426a/app-packages/kafka/src/assembly/koya-app-package.xml
----------------------------------------------------------------------
diff --git a/app-packages/kafka/src/assembly/koya-app-package.xml b/app-packages/kafka/src/assembly/koya-app-package.xml
index 9d984c1..10f807f 100644
--- a/app-packages/kafka/src/assembly/koya-app-package.xml
+++ b/app-packages/kafka/src/assembly/koya-app-package.xml
@@ -48,8 +48,8 @@
<fileMode>0755</fileMode>
</file>
<file>
- <source>${kafka.src}</source>
- <destName>${kafka.version}.tgz</destName>
+ <source>${pkg.src}/${pkg.name}</source>
+ <destName>${pkg.name}</destName>
<outputDirectory>/package/files/</outputDirectory>
<fileMode>0755</fileMode>
</file>