You are viewing a plain text version of this content. The canonical link for it is here.
Posted to s4-commits@incubator.apache.org by mm...@apache.org on 2012/07/26 10:52:41 UTC
[5/16] Update license for related projects + removed code using LGPL
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/example/model/README.md
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/README.md b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/README.md
deleted file mode 100644
index 0991e74..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/README.md
+++ /dev/null
@@ -1,390 +0,0 @@
-# Modeling Framework for S4
-
-## The Pattern Recognition Problem
-
-In this example we show how to design a pluggable probabilistic modeling framework in S4. The task is to classify incoming objects into
-well defined categories. Each object is represented by an observation vector that represents the object, also called
-features of the object. The variability and consistency of the features within a category and between categories will
-determine the accuracy of the classifier and the complexity of the models. Because it is impossible to achieve perfect accuracy in real-world systems,
-we use probabilistic models to classify the objects. So instead of just assigning a category to each object, the model will
-provide the probability that the object belongs to a category. The final decision may depend on several factors, for example,
-the cost of wrongly assign a category to an object may not be the same for all categories. In this example, we assume that
-the cost is the same for all categories and simply select the category whose model has the highest probability.
-
-Probabilistic models are widely used in many application areas. A distributed systems may be needed when decisions need to be made
-with low delay and by processing a large number of incoming events. S4 is designed to scale to an unlimited number of nodes providing
-the flexibility to run complex algorithms, process at high data rates, and deliver results with low latency. Typical applications include
-processing user events in web applications, analyzing financial market data, monitoring network traffic, and many more.
-
-To learn more, get a copy of this classic book: "Pattern Classification" by R. Duda, P. Hart, and D. Stork.
-
-## The Approach
-
-In this example we implemented an application that uses a train data set to estimate model parameters. Because most estimation
-algorithms learn iteratively, we inject the train data set several times until a stop condition is achieved. To train the models,
-we run S4 in batch mode. That is, we push the data at the highest possible rate and when a queue fills up, we let the system block
-until more space in the queues become available. In other words, no event data is ever lost in this process. To achieve this, we remove all
-latency constraints and let the process run for as long as needed until all the data is processed. This approach is quite similar
-to MapReduce except that the data is injected sequentially from a single source.
-
-Once the model parameters are estimated we are ready to run a test. In real-time applications, we would have no control over the speed of the
-incoming data vectors. If we didn't have sufficient computing resources to process all the data within the time constraints, we would be
-forced to either lose some of the data (load shedding) or switch to a less complex classification algorithm. In this example, we simply assume
-that there is no data loss.
-
-## The Forest Cover Data Set
-
-To evaluate the application we use a [publicly available labeled data set to predict
-forest cover type](http://kdd.ics.uci.edu/databases/covertype/covertype.html).
-For details about the data set please download the paper published by the author of this work.
-([PDF](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.128.2475&rep=rep1&type=pdf))
-
-Here is a description of the data. There are 7 types of forest cover. Each observation vector represents an area of
-forest with a type of cover and ten measurements (Elevation, Aspect, Slope, etc.) There is a total of 581,012 observation
-vectors in the data set.
-
-<pre>
-Name Data Type Measurement Description
-
-Cover_Type (7 types) integer 1 to 7 Forest Cover Type designation
-Elevation quantitative meters Elevation in meters
-Aspect quantitative azimuth Aspect in degrees azimuth
-Slope quantitative degrees Slope in degrees
-Horizontal_Distance_To_Hydrology quantitative meters Horz Dist to nearest surface water features
-Vertical_Distance_To_Hydrology quantitative meters Vert Dist to nearest surface water features
-Horizontal_Distance_To_Roadways quantitative meters Horz Dist to nearest roadway
-Hillshade_9am quantitative 0 to 255 index Hillshade index at 9am, summer solstice
-Hillshade_Noon quantitative 0 to 255 index Hillshade index at noon, summer soltice
-Hillshade_3pm quantitative 0 to 255 index Hillshade index at 3pm, summer solstice
-Horizontal_Distance_To_Fire_Points quantitative meters Horz Dist to nearest wildfire ignition points
-
-
-Forest Cover Types:
- 1 -- Spruce/Fir
- 2 -- Lodgepole Pine
- 3 -- Ponderosa Pine
- 4 -- Cottonwood/Willow
- 5 -- Aspen
- 6 -- Douglas-fir
- 7 -- Krummholz
-
-Class Distribution
-
-Number of records of Spruce-Fir 211840
-Number of records of Lodgepole Pine 283301
-Number of records of Ponderosa Pine 35754
-Number of records of Cottonwood/Willow 2747
-Number of records of Aspen 9493
-Number of records of Douglas-fir 17367
-Number of records of Krummholz 20510
-
-Total records 581012
-</pre>
-
-Here are the steps I used to download and prepare the data files. The files are located in the project under
-[src/main/resources/](https://github.com/leoneu/s4-piper/tree/master/src/main/resources).
-
- # Download data set and uncoompress.
- wget http://kdd.ics.uci.edu/databases/covertype/covtype.data.gz
- gunzip covtype.data.gz
-
- # Remove some columns and put the class label in the first column.
- gawk -F "," '{print $55, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10}' covtype.data > covtype-modified.data
-
- # Randomize data set.
- sort -R covtype-modified.data > covtype-random.data
-
- # Check number of data points.
- wc -l covtype-*
- # 581012 covtype-modified.data
- # 581012 covtype-random.data
-
- # Create a train and a test set.
- tail -100000 covtype-random.data > covtype-test.data
- head -481012 covtype-random.data > covtype-train.data
-
- wc -l covtype-{train,test}.data
- # 481012 covtype-train.data
- # 100000 covtype-test.data
- # 581012 total
-
-## Implementation of the Probabilistic Models
-
-The modeling package [org.apache.s4.model](https://github.com/leoneu/s4-piper/tree/master/src/main/java/io/s4/model) provides
-a basic interface for implementing probabilistic models. The main methods are:
-
-* **update()** - updates sufficient statistics given an observation vector.
-* **estimate()** - estimates model parameters using the sufficient statistics.
-* **prob()** - computes the probability.
-
-The abstract class [org.apache.s4.model.Model](https://github.com/leoneu/s4-piper/tree/master/src/main/java/io/s4/model/Model.java)
-is used to build the models. There are no dependencies with any specific implementation of a
-model because the code only uses Model to estimate and run the classifier.
-
-There are two concrete implementation of Model:
-
- * [Gaussian Model](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/model/GaussianModel.java) a single
- Gaussian distribution with parameters _mean_ and _variance_.
- * [Gaussian Mixture Model](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/model/GaussianMixtureModel.java)
- A mixture of Gaussian distributions with parameters _mixture weights_ and each Gaussian component with parameters _mean_ and
- _variance_.
-
-More model can be implemented and thanks to object oriented design, swapping models is as easy as
-editing one line of code in the [Guice](http://code.google.com/p/google-guice/)
-[Module](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/Module.java).
-
-When deployed in a cluster the same code will be run on many physical servers without changing any application code.
-As an application developer you don't have to worry about how distributed processing works. Scientist can focus on writing
-model code and dropping it in the right place. Moreover, the same code can be used to run experiments in batch mode and to
-deploy in a real-time production environment.
-
-## Training
-
-Here is the basic structure of the program:
-
-* Determine number of train vectors, and number of categories from the
- train data set. [org.apache.s4.example.model.Controller](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/Controller.java)
-* Create events of type ObsEvent and inject them into ModelPE with key = classId (classId refers to the category)
-* There is a [ModelPE](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/ModelPE.java)
- instance for each cover type (a total of seven cover types in this example with classId: 0-6).
- For each observation vector that matches the cover type, call the update() method in the Model.
-* Once all the train events are received, estimate the parameters for each model.
-
-
-We choose to use events of type ObsEvent to communicate between Processing Elements and ResultEvent to send the results to the MetricsPE.
-The events are immutable and can only be created using the constructor. The ObsEvent fields are:
-
-* _obsVector_ is the observation vector. The size of the float array should be the same for all the vectors.
-* _prob_ is the probability of the vector given the model.
-* _index_ is a unique identifier for the event.
-* _classId_ is the true category for the vector as it was labeled in the original data set.
-* _hypId_ is the hypothesized category for the vector returned by classification algorithm.
-* _isTraining_ is a boolean to differentiate between train and test modes.
-
-Here is a snippet of ObsEvent.java:
-
- public class ObsEvent extends Event {
-
- final private float[] obsVector;
- final private float prob;
- final private long index;
- final private int classId;
- final private int hypId;
-
- public ObsEvent(long index, float[] obsVector, float prob, int classId,
- int hypId) {
- this.obsVector = obsVector;
- this.prob = prob;
- this.index = index;
- this.classId = classId;
- this.hypId = hypId;
- }
-
-
-The application graph is defined in [MyApp](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/MyApp.java)
-To define a graph we simply connect processing elements to streams. When teh objects in the graph are created, they are simply a representation
-of the application. The real work is done by the processing element instances that may reside anywhere in the cluster.
-
-Notice that the application graph has a cycle (events go from ModelPE to MaximizerPE and back). This creates a minor challenge to create the
-application graph. To solve this problem we added a setter method to set the distanceStream in ClusterPE.
-
-## Testing
-
-For testing we follow the following steps:
-
-* Compute the posterior probability of the observations for each model.
-* For an observation, select model id with the highest posterior probability. This is done by MaximizerPE: for each observation
- get all seven probabilities (one from each model), select the category with the highest probability. Self destroy the instance of MaximizerPE
- once the work is done.
-* MaximizerPE sends ObsEvent with HypID back to the ModelPE instance using ClassID as key.
-* Send results to MetricsPE.
-* MetricsPE accumulate counts to compute the confusion matrix and the overall accuracy. In the confusion matrix rows correspond
- to true categories and columns to hypotheses. The diagonal
- shows the percentage of observations that were correctly categorized and the off-diagonal numbers are the errors.
-
-## Experiments
-
-We first run the classifier using the Gaussian model, that is, we model each class using a Gaussian probability density
-function for which we need to estimate its parameters (mean and variance).
-
-To run the experiment, we bind the Model type to the GaussianModel class in Module.java as follows:
-
-<pre>
- protected void configure() {
- if (config == null)
- loadProperties(binder());
-
- int vectorSize = config.getInt("model.vector_size");
- int numGaussians = config.getInt("model.num_gaussians");
-
- bind(Model.class).toInstance(
- new org.apache.s4.model.GaussianModel(vectorSize, true));
-
- }
-</pre>
-
-With this binding the GaussianModel instance will be injected in the Controller constructor.
-
-Next, we edit the properties file as follows:
-
- model.train_data = /covtype-train.data.gz
- model.test_data = /covtype-test.data.gz
- model.logger.level = DEBUG
- model.num_iterations = 1
- model.vector_size = 10
- model.output_interval_in_seconds = 2
-
-In the properties file we configure the data sets for training and testing, the logger level, the number of iterations
-(we only need one iteration to estimate the mean and variance), the vector size which is 10 for this data set and
-how often we want to print partial results, we choose 2-second intervals. A final result will be printed by the
-controller at the end of the experiment.
-
-To run using Gradle, make sure you set the Main class in build.gradle to:
-
- mainClassName = "org.apache.s4.example.model.Main"
-
-To run the experiment type:
-
- gradlew run
-
-
- and after a few seconds we get the result:
-
- Confusion Matrix [%]:
-
- 0 1 2 3 4 5 6
- ----------------------------------------
- 0: 67.4 25.2 0.7 0.0 1.0 0.3 5.5
- 1: 24.1 65.8 4.6 0.0 2.3 1.9 1.4
- 2: 0.0 19.6 64.3 3.7 0.3 12.0 0.0
- 3: 0.0 0.4 38.6 48.8 0.0 12.2 0.0
- 4: 0.0 69.0 4.8 0.0 24.0 2.2 0.0
- 5: 0.0 18.3 47.3 2.3 0.5 31.5 0.0
- 6: 70.5 0.6 0.7 0.0 0.0 0.0 28.2
-
- Accuracy: 63.1% - Num Observations: 100000
-
-The observation vectors are correctly categorized in an independent data set at a rate of 63.1%. Note that 85% of the observations
-are in categories 0 and 1. The classifier learned this fact and relied on the prior probabilities to optimize the overall accuracy
-of the classifier. That's why accuracy is higher for these categories. For example, only 3.5% of the observations are in category 6
-so the low accuracy of 28.2% has little impact on the overall accuracy. Depending on the application, this may or may not be the right
-optimization approach.
-
-Next we, want to try the more sophisticated GaussianMixtureModel. We changed the properties file as follows:
-
- model.train_data = /covtype-train.data.gz
- model.test_data = /covtype-test.data.gz
- model.logger.level = DEBUG
- model.num_gaussians = 1
- model.num_iterations = 2
- model.vector_size = 10
- model.output_interval_in_seconds = 2
-
-Note that we are only using one Gaussian per mixture which is equivalent to using the GaussianModel so we expect the results to be identical.
-We need 2 iterations because the model uses the first pass to estimate the mean and variance of the data. This is only useful when using
-more than one mixture component.
-
-We changed the Module class as follows:
-
-<pre>
- protected void configure() {
- if (config == null)
- loadProperties(binder());
-
- int vectorSize = config.getInt("model.vector_size");
- int numGaussians = config.getInt("model.num_gaussians");
-
- bind(Model.class).toInstance(
- new org.apache.s4.model.GaussianMixtureModel(vectorSize, numGaussians,
- org.apache.s4.model.GaussianMixtureModel.TrainMethod.STEP));
- }
-</pre>
-
-and we run the experiment again:
-
- gradlew run
-
-The result is identical as expected:
-
- Confusion Matrix [%]:
-
- 0 1 2 3 4 5 6
- ----------------------------------------
- 0: 67.4 25.2 0.7 0.0 1.0 0.3 5.5
- 1: 24.1 65.8 4.6 0.0 2.3 1.9 1.4
- 2: 0.0 19.6 64.3 3.7 0.3 12.0 0.0
- 3: 0.0 0.4 38.6 48.8 0.0 12.2 0.0
- 4: 0.0 69.0 4.8 0.0 24.0 2.2 0.0
- 5: 0.0 18.3 47.3 2.3 0.5 31.5 0.0
- 6: 70.5 0.6 0.7 0.0 0.0 0.0 28.2
-
- Accuracy: 63.1% - Num Observations: 100000
-
-
-Now let's increase the number of mixture components to two Gaussian distributions per category:
-
-
- model.train_data = /covtype-train.data.gz
- model.test_data = /covtype-test.data.gz
- model.logger.level = DEBUG
- model.num_gaussians = 2
- model.num_iterations = 6
- model.vector_size = 10
- model.output_interval_in_seconds = 2
-
-
- Confusion Matrix [%]:
-
- 0 1 2 3 4 5 6
- ----------------------------------------
- 0: 66.4 27.8 0.1 0.0 1.5 0.4 3.8
- 1: 24.9 63.3 3.6 0.1 4.5 3.0 0.6
- 2: 0.0 13.1 65.7 8.2 1.0 12.0 0.0
- 3: 0.0 0.4 17.0 80.9 0.0 1.7 0.0
- 4: 5.1 50.0 3.0 0.0 37.2 4.7 0.0
- 5: 0.0 15.8 39.4 7.5 1.0 36.3 0.0
- 6: 71.4 1.1 0.0 0.0 0.0 0.0 27.6
-
- Accuracy: 62.2% - Num Observations: 100000
-
-The overall accuracy went down from 63.1% to 62.2%. However, we can see a dramatic improvement in category 3
-(from 48.8% to 80.9%) at the cost of a slight degradation in categories 0 and 1. Clearly, using two Gaussians
-per category helped category three.
-
-To improve the accuracy of the classifier, one could do some additional analysis and come up with an improved
-model until the accuracy is acceptable for the target application. For example, why are so many category 6 observations
-classified as category 0? Maybe we need a different number of mixtures per category to allocate more parameters to the
-categories with more training data and fewer to the other ones. Give it a try and let me know. I will add any
-models that get better overall accuracy than this one.
-
-## Performance
-
-I tested the execution speed on a single node configuration in a MacBook air with a Core i5 CPU and 4GB of RAM. The data
-is read from the solid state disk and uncompressed using gzip in every iteration. Initialization time is
-excluded from the measurements. Because I used only one node and all the data is local, there is no network overhead
-involved.
-
-The following results are for the GaussianMixture Model with 2 components per mixture and 6 iterations.
-
- Total training time was 33 seconds.
- Training time per observation was 69 microseconds.
- Training time per observation per iteration was 11 microseconds.
- Total testing time was 4 seconds.
- Testing time per observation was 8 microseconds.
-
-Based on this number, the data rate at which we injected data for training was (1/11 microseconds) or 90,000
-observations per second. If we look at the ObsEvent class, the effective number of bits transmitted per event is:
-
- 10 x float + 1 x float + 2 x int + 1 x long = 11 x 32 + 2 x 32 + 1 x 64 = 480 bits / observation
-
-This results in an injected data rate of 90,000 x 480 bits/sec = 43 mbps
-
-On an Ubuntu machine with an Intel Core i7-860 processor the average time per observation-iteration was
-8 microseconds or about 30% faster running at a data rate of 56 mbps.
-
-This is just to get an idea of the execution speed before even thinking about how to optimize. The throughput will vary
-greatly depending on the complexity of the algorithm and the hardware configuration.
-
-
-Please share your feedback at:
-http://groups.google.com/group/s4-project/topics
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultEvent.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultEvent.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultEvent.java
deleted file mode 100644
index 1274f82..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultEvent.java
+++ /dev/null
@@ -1,60 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.example.model;
-
-import org.apache.s4.base.Event;
-
-final public class ResultEvent extends Event {
-
- private long index;
- private int classId;
- private int hypId;
-
- public ResultEvent() {
-
- }
-
- public ResultEvent(long index, int classId,
- int hypId) {
- this.index = index;
- this.classId = classId;
- this.hypId = hypId;
- }
-
- /**
- * @return the index of the data vector.
- */
- public long getIndex() {
- return index;
- }
-
- /**
- * @return the true class of the vector.
- */
- public int getClassId() {
- return classId;
- }
-
- /**
- * @return the hypothesized class of the vector.
- */
- public int getHypId() {
- return hypId;
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultKeyFinder.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultKeyFinder.java
deleted file mode 100644
index 96147f1..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultKeyFinder.java
+++ /dev/null
@@ -1,39 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.example.model;
-
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.s4.base.KeyFinder;
-
-public class ResultKeyFinder implements KeyFinder<ResultEvent> {
-
- @Override
- public List<String> get(ResultEvent event) {
-
- List<String> results = new ArrayList<String>();
-
- /* Retrieve the user ID and add it to the list. */
- results.add("1234");
-
- return results;
- }
-
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/model/Gaussian.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/Gaussian.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/Gaussian.java
deleted file mode 100644
index c64336c..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/Gaussian.java
+++ /dev/null
@@ -1,34 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.model;
-
-import java.lang.annotation.Retention;
-import java.lang.annotation.Target;
-import static java.lang.annotation.RetentionPolicy.RUNTIME;
-import static java.lang.annotation.ElementType.PARAMETER;
-import static java.lang.annotation.ElementType.FIELD;
-import static java.lang.annotation.ElementType.METHOD;
-
-import com.google.inject.BindingAnnotation;
-
-
- @BindingAnnotation @Target({ FIELD, PARAMETER, METHOD }) @Retention(RUNTIME)
- public @interface Gaussian {}
-
-
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixture.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixture.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixture.java
deleted file mode 100644
index 8a5cb81..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixture.java
+++ /dev/null
@@ -1,34 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.model;
-
-import java.lang.annotation.Retention;
-import java.lang.annotation.Target;
-import static java.lang.annotation.RetentionPolicy.RUNTIME;
-import static java.lang.annotation.ElementType.PARAMETER;
-import static java.lang.annotation.ElementType.FIELD;
-import static java.lang.annotation.ElementType.METHOD;
-
-import com.google.inject.BindingAnnotation;
-
-
- @BindingAnnotation @Target({ FIELD, PARAMETER, METHOD }) @Retention(RUNTIME)
- public @interface GaussianMixture {}
-
-
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixtureModel.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixtureModel.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixtureModel.java
deleted file mode 100644
index 7047a7d..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixtureModel.java
+++ /dev/null
@@ -1,320 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.model;
-
-import org.apache.commons.lang.NotImplementedException;
-import org.apache.s4.util.MatrixOps;
-import org.ejml.data.D1Matrix64F;
-import org.ejml.data.DenseMatrix64F;
-import org.ejml.ops.CommonOps;
-
-
-/**
- * A multivariate Gaussian mixture model. Only diagonal covariance matrices are
- * supported.
- *
- *
- */
-public class GaussianMixtureModel extends Model {
-
- /** Supported algorithms for training this model. */
- public enum TrainMethod {
-
- /**
- * Estimate mean and variance of Gaussian distribution in the first
- * iteration. Create the target number of Gaussian components
- * (numComponents) in the mixture at the end of the first iteration
- * using the estimated mean and variance.
- */
- STEP,
-
- /**
- * Double the number of Gaussian components at the end of each
- * iteration.
- */
- DOUBLE,
-
- /** Do not allocate structures for training. */
- NONE
- }
-
- final private int numElements;
- final private TrainMethod trainMethod;
- private int numComponents;
- private double numSamples;
- private D1Matrix64F posteriorSum;
- private D1Matrix64F weights;
- private D1Matrix64F logWeights;
- private D1Matrix64F tmpProbs1;
- private D1Matrix64F tmpProbs2;
- private double totalLikelihood;
- private GaussianModel[] components;
- private int iteration = 0;
-
- public GaussianMixtureModel(int numElements, int numComponents,
- TrainMethod trainMethod) {
- super();
- this.numComponents = numComponents;
- this.numElements = numElements;
- this.trainMethod = trainMethod;
-
- if (trainMethod == TrainMethod.DOUBLE)
- throw new NotImplementedException(
- "Want this? Join as a contributor at http://s4.io");
-
- /* Allocate arrays needed for estimation. */
- isTrain = false;
- if (trainMethod != TrainMethod.NONE) {
- setTrain(true);
-
- if (trainMethod == TrainMethod.STEP) {
-
- /* Set up for first iteration using a single Gaussian. */
- allocateTrainDataStructures(1);
- }
- }
- }
-
- /*
- * TODO: we use this method when the pattern is: create a prototype of the
- * model and use it to create instances. Notice that we are allocating data
- * structures in the prototype that we will not use in this case. Not a big
- * deal but we may want to optimize this somehow later.
- */
- public Model create() {
-
- return new GaussianMixtureModel(numElements, numComponents, trainMethod);
- }
-
- private void allocateTrainDataStructures(int numComp) {
-
- components = new GaussianModel[numComp];
- for (int i = 0; i < numComp; i++) {
- this.components[i] = new GaussianModel(numElements, true);
-
- this.weights = new DenseMatrix64F(numComp, 1);
- CommonOps.set(this.weights, 1.0 / numComp);
- this.logWeights = new DenseMatrix64F(numComp, 1);
- CommonOps.set(this.logWeights, Math.log(1.0 / numComp));
- posteriorSum = new DenseMatrix64F(numComp, 1);
- tmpProbs1 = new DenseMatrix64F(numComp, 1);
- tmpProbs2 = new DenseMatrix64F(numComp, 1);
- }
- }
-
- /*
- * This method is used in {@link TrainMethod#STEP} Convert to a mixture with
- * N components. This method guarantees that the data structures are created
- * and that all the variables are set for starting a new training iteration.
- */
- private void increaseNumComponents(int newNumComponents) {
-
- /*
- * We use the Gaussian distribution of the parent GMM to create the
- * children.
- */
-
- /* Get mean and variance from parent before we allocate resized data structures. */
- D1Matrix64F mean = MatrixOps.doubleArrayToMatrix(this.components[0]
- .getMean());
- D1Matrix64F variance = MatrixOps.doubleArrayToMatrix(this.components[0]
- .getVariance());
-
- /* Throw away all previous data structures. */
- allocateTrainDataStructures(newNumComponents);
-
- /*
- * Create new mixture components. Abandon the old ones. We already got
- * the mean and variance in the previous step.
- */
- for (int i = 0; i < newNumComponents; i++) {
- components[i].setMean(MatrixOps.createRandom(i, mean, variance));
- components[i].setVariance(new DenseMatrix64F(variance));
- }
- }
-
- /* Thread safe internal logProb method. Must pass temp array. */
- private double logProbInternal(D1Matrix64F obs, D1Matrix64F probs) {
-
- /* Compute log probabilities for this observation. */
- for (int i = 0; i < components.length; i++) {
- probs.set(i, components[i].logProb(obs) + logWeights.get(i));
- }
-
- /*
- * To simplify computation, use the max prob in the denominator instead
- * of the sum.
- */
- return CommonOps.elementMax(probs);
- }
-
- public double logProb(D1Matrix64F obs) {
-
- return logProbInternal(obs, tmpProbs1);
- }
-
- public double logProb(double[] obs) {
-
- return logProb(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- public double logProb(float[] obs) {
-
- return logProb(MatrixOps.floatArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(D1Matrix64F obs) {
-
- return Math.exp(logProb(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(double[] obs) {
-
- return prob(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(float[] obs) {
-
- return prob(MatrixOps.floatArrayToMatrix(obs));
-
- }
-
- /** Update using Matrix array. */
- public void update(D1Matrix64F obs) {
-
- if (isTrain() == true) {
-
- /* Compute log probabilities for this observation. */
- double maxProb = logProbInternal(obs, tmpProbs2);
- totalLikelihood += maxProb;
- CommonOps.add(tmpProbs2, -maxProb);
-
- /* Compute posterior probabilities. */
- MatrixOps.elementExp(tmpProbs2);
-
- /* Update posterior sum, needed to compute mixture weights. */
- CommonOps.addEquals(posteriorSum, tmpProbs2);
-
- for (int i = 0; i < components.length; i++) {
- components[i].update(obs, tmpProbs2.get(i));
- }
-
- /* Count number of observations. */
- numSamples++;
- }
- }
-
- public void update(double[] obs) {
- update(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /** Update using float array. */
- public void update(float[] obs) {
- update(MatrixOps.floatArrayToMatrix(obs));
- }
-
- @Override
- public void estimate() {
-
- if (isTrain() == true) {
-
- /* Estimate mixture weights. */
- // double sum = CommonOps.elementSum(posteriorSum);
- // CommonOps.scale(1.0/sum, posteriorSum, weights);
- CommonOps.scale(1.0 / numSamples, posteriorSum, weights);
- MatrixOps.elementLog(weights, logWeights);
-
- /* Estimate component density. */
- for (int i = 0; i < components.length; i++) {
- components[i].estimate();
- }
-
- /*
- * After the first iteration, we can estimate the target number of
- * mixture components.
- */
- if (iteration == 0 && trainMethod == TrainMethod.STEP) {
-
- increaseNumComponents(numComponents);
- }
- iteration++;
- }
- }
-
- @Override
- public void clearStatistics() {
- if (isTrain() == true) {
-
- for (int i = 0; i < components.length; i++) {
- components[i].clearStatistics();
- }
- CommonOps.set(posteriorSum, 0.0);
- numSamples = 0;
- totalLikelihood = 0;
- }
-
- }
-
- /** Number of Gaussian components in the mixture. */
- public int getNumComponents() {
- return this.numComponents;
- }
-
- /** Data vector size. */
- public int getNumElements() {
- return this.numElements;
- }
-
- /**
- * @return the value of the parameters and sufficient statistics of this
- * model in a printable format.
- */
- public String toString() {
-
- StringBuilder sb = new StringBuilder("");
- sb.append("Gaussian Mixture Model\n");
- sb.append("num samples: " + numSamples + "\n");
- sb.append("num components: " + components.length + "\n");
- sb.append("weights: " + weights.toString() + "\n");
- sb.append("log weights: " + logWeights.toString() + "\n");
- sb.append("total log likelihood: " + totalLikelihood + "\n");
-
- for (int i = 0; i < components.length; i++) {
- sb.append(components[i].toString());
- }
-
- return sb.toString();
- }
-
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianModel.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianModel.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianModel.java
deleted file mode 100644
index 6f86deb..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianModel.java
+++ /dev/null
@@ -1,383 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.model;
-
-
-import org.apache.s4.util.MatrixOps;
-import org.ejml.data.D1Matrix64F;
-import org.ejml.data.DenseMatrix64F;
-import org.ejml.ops.CommonOps;
-
-/**
- * A multivariate Gaussian model with parameters mean (mu) and variance (sigma
- * squared). Only diagonal covariance matrices are supported.
- *
- *
- */
-public class GaussianModel extends Model {
-
- public final static double SMALL_VARIANCE = 0.01f;
- public final static double minNumSamples = 0.01f;
-
- private boolean isDiagonal = true; // Full covariance not yet supported.
- private D1Matrix64F sumx;
- private D1Matrix64F sumxsq;
- private D1Matrix64F tmpArray;
- private double numSamples;
- private D1Matrix64F mean;
- private D1Matrix64F variance; // ==> sigma squared
- private int numElements;
-
- private double const1; // -(N/2)log(2PI) Depends only on numElements.
- private double const2; // const1 - sum(log sigma_i) Also depends on
- // variance.
-
- /**
- * @param numElements
- * the model dimension.
- * @param train
- * allocate training arrays when true.
- */
- public GaussianModel(int numElements, boolean train) {
- this(numElements, null, null, train);
- }
-
- /**
- * Initialize model, no allocation of training arrays.
- *
- * @param numElements
- * the model dimension.
- * @param mean
- * model parameter.
- * @param variance
- * model parameter.
- */
- public GaussianModel(int numElements, D1Matrix64F mean, D1Matrix64F variance) {
- this(numElements, mean, variance, false);
- }
-
- /**
- * Initialize model, no allocation of training arrays.
- *
- * @param numElements
- * the model dimension.
- * @param mean
- * model parameter.
- * @param variance
- * model parameter.
- * @param train
- * allocate training arrays when true.
- */
- public GaussianModel(int numElements, D1Matrix64F mean,
- D1Matrix64F variance, boolean train) {
- super();
- this.numElements = numElements;
- tmpArray = new DenseMatrix64F(numElements, 1);
-
- if (mean == null) {
- this.mean = new DenseMatrix64F(numElements, 1);
- } else {
- this.mean = mean;
- }
-
- if (variance == null) {
- this.variance = new DenseMatrix64F(numElements, 1);
- CommonOps.set(this.variance, SMALL_VARIANCE);
- } else {
- this.variance = variance;
- }
-
- const1 = -numElements * (float) Math.log(2 * Math.PI) / 2;
- MatrixOps.elementLog(this.variance, tmpArray);
- const2 = const1 - CommonOps.elementSum(tmpArray) / 2.0;
-
- /* Allocate arrays needed for estimation. */
- if (train == true) {
- setTrain(true);
- sumx = new DenseMatrix64F(numElements, 1);
- sumxsq = new DenseMatrix64F(numElements, 1);
- clearStatistics();
- } else {
- setTrain(false);
- }
-
- }
-
- public Model create() {
-
- return new GaussianModel(numElements, isTrain);
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the log probability.
- */
- public double logProb(D1Matrix64F obs) {
-
- CommonOps.sub(mean, obs, tmpArray);
- MatrixOps.elementSquare(tmpArray);
- CommonOps.elementDiv(tmpArray, variance);
- return const2 - CommonOps.elementSum(tmpArray) / 2.0;
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the log probability.
- */
- public double logProb(float[] obs) {
-
- return logProb(MatrixOps.floatArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the log probability.
- */
- public double logProb(double[] obs) {
-
- return logProb(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#evaluate(double[])
- */
- public double prob(double[] obs) {
-
- return prob(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /** Evaluate using float array. */
- public double prob(float[] obs) {
-
- return prob(MatrixOps.floatArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(D1Matrix64F obs) {
-
- return Math.exp(logProb(obs));
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#update(double[])
- */
- public void update(double[] obs) {
-
- update(MatrixOps.doubleArrayToMatrix(obs));
-
- }
-
- /** Update using float array. */
- public void update(float[] obs) {
-
- update(MatrixOps.floatArrayToMatrix(obs));
-
- }
-
- /**
- * Update sufficient statistics.
- *
- * @param obs
- * the observed data vector.
- */
- public void update(D1Matrix64F obs) {
-
- if (isTrain() == true) {
-
- /* Update sufficient statistics. */
- CommonOps.add(obs, sumx, sumx);
- MatrixOps.elementSquare(obs, tmpArray);
- CommonOps.add(tmpArray, sumxsq, sumxsq);
- numSamples++;
- }
- }
-
- /**
- * Update sufficient statistics.
- *
- * @param obs
- * the observed data vector.
- * @param weight
- * the weight assigned to this observation.
- */
- public void update(D1Matrix64F obs, double weight) {
-
- if (isTrain() == true) {
-
- /* Update sufficient statistics. */
- CommonOps.scale(weight, obs, tmpArray);
- CommonOps.add(tmpArray, sumx, sumx);
-
- MatrixOps.elementSquare(obs, tmpArray);
- CommonOps.scale(weight, tmpArray);
- CommonOps.add(tmpArray, sumxsq, sumxsq);
-
- numSamples += weight;
- }
-
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#estimate()
- */
- public void estimate() {
-
- if (numSamples > minNumSamples) {
-
- /* Estimate the mean. */
- CommonOps.scale(1.0 / numSamples, sumx, mean);
-
- /*
- * Estimate the variance. sigma_sq = 1/n (sumxsq - 1/n sumx^2) or
- * 1/n sumxsq - mean^2.
- */
- D1Matrix64F tmp = variance; // borrow as an intermediate array.
-
- MatrixOps.elementSquare(mean, tmpArray);
- CommonOps.scale(1.0 / numSamples, sumxsq, tmp);
-
- CommonOps.sub(tmp, tmpArray, variance);
-
- MatrixOps.elementFloor(SMALL_VARIANCE, variance, variance);
-
- } else {
-
- /* Not enough training sample. */
- CommonOps.set(variance, SMALL_VARIANCE);
- CommonOps.set(mean, 0.0);
- }
-
- /* Update log Gaussian constant. */
- MatrixOps.elementLog(this.variance, tmpArray);
- const2 = const1 - CommonOps.elementSum(tmpArray) / 2.0;
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#clearStatistics()
- */
- public void clearStatistics() {
-
- if (isTrain() == true) {
- CommonOps.set(sumx, 0.0);
- CommonOps.set(sumxsq, 0.0);
- numSamples = 0;
- }
- }
-
- /** @return the mean (mu) of the Gaussian density. */
- public double[] getMean() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(mean);
- return tmp.getData();
- }
-
- /** @return the variance (sigma squared) of the Gaussian density. */
- public double[] getVariance() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(variance);
- return tmp.getData();
- }
-
- public void setMean(D1Matrix64F mean) {
- this.mean = mean;
- }
-
- public void setVariance(D1Matrix64F variance) {
- this.variance = variance;
-
- /* Update log Gaussian constant. */
- MatrixOps.elementLog(this.variance, tmpArray);
- const2 = const1 - CommonOps.elementSum(tmpArray) / 2.0;
- }
-
- /** @return the standard deviation (sigma) of the Gaussian density. */
- public double[] getStd() {
-
- DenseMatrix64F std = new DenseMatrix64F(numElements, 1);
- MatrixOps.elementSquareRoot(variance, std);
- return std.getData();
- }
-
- /** @return the sum of the observed vectors. */
- public double[] getSumX() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(sumx);
- return tmp.getData();
- }
-
- /** @return the sum of the observed vectors squared. */
- public double[] getSumXSq() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(sumxsq);
- return tmp.getData();
- }
-
- /** @return the number of observations. */
- public double getNumSamples() {
-
- return numSamples;
- }
-
- /** @return the dimensionality. */
- public int getNumElements() {
-
- return numElements;
- }
-
- /** @return true if the covariance matrix is diagonal. */
- public boolean isDiagonal() {
-
- return isDiagonal;
- }
-
- /**
- * @return the value of the parameters and sufficient statistics of this
- * model in a printable format.
- */
- public String toString() {
-
- StringBuilder sb = new StringBuilder("");
- sb.append("Gaussian Model\n");
- sb.append("const: " + const2 + "\n");
-
- sb.append("num samp: " + numSamples + "\n");
-
- sb.append("mean: " + mean.toString() + "\n");
- sb.append("var: " + variance.toString() + "\n");
- sb.append("sumx: " + sumx.toString() + "\n");
- sb.append("sunxsq: " + sumxsq.toString() + "\n");
-
- return sb.toString();
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/model/Model.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/Model.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/Model.java
deleted file mode 100644
index 68d3cf0..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/Model.java
+++ /dev/null
@@ -1,97 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.model;
-
-/**
- *
- * Base class for statistical models.
- *
- *
- */
-abstract public class Model {
-
- protected boolean isTrain;
- protected String name;
-
- public Model() {
- }
-
- public Model(String name, boolean isTrain) {
- this.name = name;
- this.isTrain = isTrain;
- }
-
- /**
- * Return an instance of this model initialized with the same parameters as its parent.
- *
- * @return a copy of the parent model.
- */
- abstract public Model create();
-
- /**
- * Compute the probability of the observed vector.
- *
- * @param obs
- * An observed data vector.
- * @return the probability.
- */
- abstract public double prob(float[] obs);
-
- /**
- * Compute the log probability of the observed vector.
- *
- * @param obs
- * An observed data vector.
- * @return the log probability.
- */
- abstract public double logProb(float[] obs);
-
- /**
- * Update sufficient statistics/
- *
- * @param obs
- * An observed data vector.
- */
- abstract public void update(float[] obs);
-
- /** Estimate model parameters. */
- abstract public void estimate();
-
- /** Clear sufficient statistics. */
- abstract public void clearStatistics();
-
- /** @return true if training resources have been initialized. */
- public boolean isTrain() {
- return isTrain;
- }
-
- public void setTrain(boolean isTrain) {
- this.isTrain = isTrain;
- }
-
- /** @return model name. */
- public String getName() {
- return name;
- }
-
- /** Set model name. */
- public void setName(String name) {
- this.name = name;
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/java/org/apache/s4/util/MatrixOps.java
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/util/MatrixOps.java b/subprojects/s4-example/src/main/java/org/apache/s4/util/MatrixOps.java
deleted file mode 100644
index c56f39a..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/util/MatrixOps.java
+++ /dev/null
@@ -1,285 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.util;
-
-import java.util.Random;
-
-import org.ejml.data.D1Matrix64F;
-import org.ejml.data.DenseMatrix64F;
-
-/**
- * Extensions to the EJML library.
- *
- */
-public class MatrixOps {
-
- /**
- * <p>
- * Performs an in-place element by element natural logarithm operation.<br>
- * <br>
- * a<sub>ij</sub> = log(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the log. Modified.
- */
- public static void elementLog(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, Math.log(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an element by element natural logarithm operation.<br>
- * <br>
- * b<sub>ij</sub> = log(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the log. Not Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementLog(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, Math.log(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an in-place element by element exponential function.<br>
- * <br>
- * a<sub>ij</sub> = exp(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the exponentiation. Modified.
- */
- public static void elementExp(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, Math.exp(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an element by element natural exponential function.<br>
- * <br>
- * b<sub>ij</sub> = exp(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the exponentiation. Not
- * Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementExp(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, Math.exp(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an in-place element by element square operation.<br>
- * <br>
- * a<sub>ij</sub> = a<sub>ij</sub>^2
- * </p>
- *
- * @param a
- * The matrix on which we perform the square. Modified.
- */
- public static void elementSquare(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, a.get(i) * a.get(i));
- }
- }
-
- /**
- * <p>
- * Performs an element by element square.<br>
- * <br>
- * b<sub>ij</sub> = a<sub>ij</sub>^2
- * </p>
- *
- * @param a
- * The matrix on which we perform the square. Not Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementSquare(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, a.get(i) * a.get(i));
- }
- }
-
- /**
- * <p>
- * Performs an in-place element by element square root operation.<br>
- * <br>
- * a<sub>ij</sub> = sqrt(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the square root. Modified.
- */
- public static void elementSquareRoot(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, Math.sqrt(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an element by element square root.<br>
- * <br>
- * b<sub>ij</sub> = sqrt(<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the square root. Not Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementSquareRoot(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, Math.sqrt(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Set a floor value for the matrix elements.<br>
- *
- * @param floor
- * The minimum element value.
- * @param a
- * The input matrix. Not Modified.
- * @param b
- * Matrix whose elements with values less than floor are set to
- * floor. Modified.
- */
- public static void elementFloor(double floor, D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- if (a.get(i) < floor)
- b.set(i, floor);
- else
- b.set(i, a.get(i));
- }
- }
-
- /** Convert an array of doubles to a matrix. A new matrix is created. */
- public static D1Matrix64F doubleArrayToMatrix(double[] data) {
-
- DenseMatrix64F tmp = new DenseMatrix64F(data.length, 1);
- for (int i = 0; i < data.length; i++) {
- tmp.set(i, data[i]);
- }
- return tmp;
- }
-
- /** Convert an array of doubles to a matrix passed as a parameter. */
- public static D1Matrix64F doubleArrayToMatrix(double[] data, D1Matrix64F mat) {
-
- for (int i = 0; i < data.length; i++) {
- mat.set(i, data[i]);
- }
- return mat;
- }
-
- /** Convert an array of floats to a matrix. A new matrix is created. */
- public static D1Matrix64F floatArrayToMatrix(float[] data) {
-
- DenseMatrix64F tmp = new DenseMatrix64F(data.length, 1);
- for (int i = 0; i < data.length; i++) {
- tmp.set(i, data[i]);
- }
- return tmp;
- }
-
- /** Convert an array of doubles to a matrix passed as a parameter. */
- public static D1Matrix64F floatArrayToMatrix(float[] data, D1Matrix64F mat) {
-
- for (int i = 0; i < data.length; i++) {
- mat.set(i, data[i]);
- }
- return mat;
- }
-
- /**
- * Create a random vector using the mean and variance of a Gaussian
- * distribution.
- */
- public static D1Matrix64F createRandom(long seed, D1Matrix64F mean, D1Matrix64F variance) {
- if (mean.numRows != variance.numRows || mean.numCols != variance.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = mean.getNumElements();
- DenseMatrix64F mat = new DenseMatrix64F(size, 1);
- Random random = new Random(seed);
-
- for (int i = 0; i < size; i++) {
- mat.set(i, mean.get(i) + random.nextGaussian() * Math.sqrt(variance.get(i)));
- }
-
- return mat;
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/resources/covtype-test.data.gz
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/resources/covtype-test.data.gz b/subprojects/s4-example/src/main/resources/covtype-test.data.gz
deleted file mode 100644
index ec7462c..0000000
Binary files a/subprojects/s4-example/src/main/resources/covtype-test.data.gz and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/resources/covtype-train-1000.data.gz
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/resources/covtype-train-1000.data.gz b/subprojects/s4-example/src/main/resources/covtype-train-1000.data.gz
deleted file mode 100644
index 5be2362..0000000
Binary files a/subprojects/s4-example/src/main/resources/covtype-train-1000.data.gz and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/resources/covtype-train.data.gz
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/resources/covtype-train.data.gz b/subprojects/s4-example/src/main/resources/covtype-train.data.gz
deleted file mode 100644
index 44ccd84..0000000
Binary files a/subprojects/s4-example/src/main/resources/covtype-train.data.gz and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/resources/model.properties
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/resources/model.properties b/subprojects/s4-example/src/main/resources/model.properties
deleted file mode 100644
index bf627a5..0000000
--- a/subprojects/s4-example/src/main/resources/model.properties
+++ /dev/null
@@ -1,14 +0,0 @@
-# Use the training set with 1000 data point for development.
-# model.train_data = /covtype-train.data.gz
-# model.train_data = /covtype-train-1000.data.gz
-#
-model.train_data = /covtype-train-1000.data.gz
-model.test_data = /covtype-test.data.gz
-model.logger.level = DEBUG
-model.num_gaussians = 2
-model.num_iterations = 6
-model.vector_size = 10
-# model.output_interval_in_events = 10
-model.output_interval_in_seconds = 2
-
-
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/subprojects/s4-example/src/main/resources/s4-piper-example.properties
----------------------------------------------------------------------
diff --git a/subprojects/s4-example/src/main/resources/s4-piper-example.properties b/subprojects/s4-example/src/main/resources/s4-piper-example.properties
deleted file mode 100644
index b60f40a..0000000
--- a/subprojects/s4-example/src/main/resources/s4-piper-example.properties
+++ /dev/null
@@ -1,7 +0,0 @@
-comm.queue_emmiter_size = 8000
-comm.queue_listener_size = 8000
-cluster.hosts = localhost
-cluster.ports = 5077
-cluster.lock_dir = /tmp
-
-
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/test-apps/simple-deployable-app-2/build.gradle
----------------------------------------------------------------------
diff --git a/test-apps/simple-deployable-app-2/build.gradle b/test-apps/simple-deployable-app-2/build.gradle
deleted file mode 100644
index 17c4db4..0000000
--- a/test-apps/simple-deployable-app-2/build.gradle
+++ /dev/null
@@ -1,245 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/**
-* Apache S4 Application Build File
-*
-* Use this script to buils and package S4 apps.
-*
-* Run 'gradle install' on the s4 project to publish to your local maven repo.
-*
-* TODO: This should probably be distributed as an s4 plugin for Gradle.
-* TODO: There seem to be to be similarities with the war and jetty plugins. (war -> s4r, jetty -> s4Run).
-* We should make it easy to test the app from this script by a running a test task that starts and stops
-* an s4 server. See: http://www.gradle.org/releases/1.0-milestone-3/docs/userguide/userguide_single.html#war_plugin
-*
-* This is an interesting discussion:
-* http://gradle.1045684.n5.nabble.com/Exclude-properties-file-from-war-td3365147.html
-*
-*/
-
-/* Set the destination where we want to install the apps. */
-//s4AppInstallDir = "/tmp/s4Apps" // TODO: decide how to standarize dirs, use env var?
-
-project.ext["s4AppInstallDir"] = hasProperty('appsDir') ? "$appsDir" : "/tmp/appsDir"
-
-project.ext["s4Version"] = '0.5.0-SNAPSHOT'
-description = 'Apache S4 App'
-//defaultTasks 'installS4R'
-project.ext["archivesBaseName"] = "$project.name"
-project.ext["distRootFolder"] = "$archivesBaseName-${-> version}"
-
-
-// Append the suffix 'SNAPSHOT' when the build is not for release.
-version = new Version(major: 0, minor: 0, bugfix: 0, isRelease: false)
-group = 'org.apache.s4'
-
-apply plugin: 'java'
-apply plugin: 'eclipse'
-
-/* The app classname is set automatically from the source files. */
-def appClassname = ''
-
-/* Set Java version. */
-sourceCompatibility = 1.6
-targetCompatibility = 1.6
-
-
-
-/* All project libraries must be defined here. */
-project.ext["libraries"] = [
- json: 'org.json:json:20090211',
- guava: 'com.google.guava:guava:10.0.1',
- gson: 'com.google.code.gson:gson:1.6',
- guice: 'com.google.inject:guice:3.0',
- guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
- guice_grapher: 'com.google.inject:guice-grapher:3.0',
- flexjson: 'net.sf.flexjson:flexjson:2.1',
- bcel: 'org.apache.bcel:bcel:5.2',
- jakarta_regexp: 'jakarta-regexp:jakarta-regexp:1.4',
- kryo: 'com.googlecode:kryo:1.04',
- netty: 'org.jboss.netty:netty:3.2.5.Final',
- reflectasm: 'com.esotericsoftware:reflectasm:0.8',
- minlog: 'com.esotericsoftware:minlog:1.2',
- asm: 'asm:asm:3.2',
- commons_io: 'commons-io:commons-io:2.0.1',
- commons_config: 'commons-configuration:commons-configuration:1.6',
- commons_codec: 'commons-codec:commons-codec:1.4',
- commons_httpclient: 'commons-httpclient:commons-httpclient:3.1',
- commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
- slf4j: 'org.slf4j:slf4j-api:1.6.1',
- logback_core: 'ch.qos.logback:logback-core:0.9.29',
- logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
- zk: 'org.apache.zookeeper:zookeeper:3.3.1',
- jcip: 'net.jcip:jcip-annotations:1.0',
- junit: 'junit:junit:4.10',
- ]
-
-
-dependencies {
-
- /* S4 Platform. We only need the API, not the transitive dependencies. */
-// s4Libs.each { module ->
-// compile( module ) //{ transitive = false }
-// s4API( module )
-// }
-
- compile project(":s4-base")
- compile project(":s4-comm")
- compile project(":s4-core")
-
- /* Logging. */
- compile( libraries.slf4j )
- compile( libraries.logback_core )
- compile( libraries.logback_classic )
-
- /* Commons. */
- compile( libraries.commons_io )
- compile( libraries.commons_config )
- compile( libraries.commons_coll )
-
- /* Misc. */
- compile( libraries.jcip )
-
- /* Testing. */
- testCompile( libraries.junit )
-}
-
-/* Set the manifest attributes for the S4 archive here.
-* TODO: separate custom properties from std ones and set custom properties at the top of the build script.
-*/
-manifest.mainAttributes(
- provider: 'gradle',
- 'Implementation-Url': 'http://incubator.apache.org/projects/s4.html',
- 'Implementation-Version': version,
- 'Implementation-Vendor': 'Apache S4',
- 'Implementation-Vendor-Id': 's4app',
- 'S4-App-Class': appClassname, // gets set by the s4r task.
- 'S4-Version': s4Version
- )
-
-project.ext["appDependencies"] = ( configurations.compile )
-
-
-
-
-
-
-task copyDependenciesToLib(type: Copy) {
- into project.libsDir.path+"/lib"
- from configurations.runtime
-}
-
-task buildProjectJar() {
- dependsOn jar {
- destinationDir file(project.libsDir.path + "/app")
- from sourceSets.main.output
- }
-}
-
-
-
-/* This task will extract all the class files and create a fat jar. We set the manifest and the extension to make it an S4 archive file. */
-// TODO: exclude schenma files as needed (not critical) see: http://forums.gradle.org/gradle/topics/using_gradle_to_fat_jar_a_spring_project
-task s4r(type: Jar) {
- dependsOn cleanCopyDependenciesToLib, copyDependenciesToLib, cleanBuildProjectJar, buildProjectJar
- from { project.libsDir }
- manifest = project.manifest
- extension = 's4r'
-
- /* Set class name in manifest. Parse source files until we find a class that extends App.
- * Get fully qualified Java class name and set attribute in Manifest.
- */
- sourceSets.main.allSource.files.each { File file ->
- if (appClassname =="" || appClassname == "UNKNOWN") {
- // only execute the closure for this file if we haven't already found the app class name
- appClassname = getAppClassname(file)
- if(appClassname != "") {
- manifest.mainAttributes('S4-App-Class': appClassname)
- }
- }
- }
-
- if (appClassname == "UNKNOWN") {
-
- println "Couldn't find App class in source files...aborting."
- exit(1)
- }
-}
-
-/* List the artifacts that will br added to the s4 archive (and explode if needed). */
-s4r << {
- appDependencies.each { File file -> println 'Adding to s4 archive: ' + file.name }
- configurations.archives.allArtifacts.files.each { File file -> println 'Adding to s4 archive: ' + file.name }
-
- /* This is for debugging. */
- //configurations.s4All.each { File file -> println 's4All: ' + file.name }
- //deployableDependencies.each { File file -> println 'Deploy: ' + file.name }
-
- // more debugging statements.
- //sourceSets.main.compileClasspath.each { File file -> println 'compileClasspath: ' + file.name }
-
-}
-
-/* Install the S4 archive to the install directory. */
-task installS4R (type: Copy) {
- dependsOn s4r
- from s4r.archivePath
- into s4AppInstallDir
-}
-
-/* Generates the gradlew scripts.
-http://www.gradle.org/1.0-milestone-3/docs/userguide/gradle_wrapper.html */
-task wrapper(type: Wrapper) { gradleVersion = '1.0-milestone-3' }
-
-
-/* Parse source file to get the app classname so we can use it in the manifest.
-* TODO: Use a real Java parser. (This is not skippong comments for example.)
-*/
-def getAppClassname(file) {
- def classname = "UNKNOWN"
- def lines= file.readLines()
- def packageName = ""
- for(line in lines) {
-
- def pn = line =~ /.*package\s+([\w\.]+)\s*;.*/
- if(pn) {
- packageName = pn[0][1] + "."
- }
-
- def an = line =~ /.*public\s+class\s+(\w+)\s+extends.+App.*\{/
- if (an) {
- classname = packageName + an[0][1]
- println "Found app class name: " + classname
- break
- }
- }
- classname
-}
-
-class Version {
- int major
- int minor
- int bugfix
- boolean isRelease
-
- String toString() {
- "$major.$minor.$bugfix${isRelease ? '' : '-SNAPSHOT'}"
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/A.java
----------------------------------------------------------------------
diff --git a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/A.java b/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/A.java
deleted file mode 100644
index f84d7db..0000000
--- a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/A.java
+++ /dev/null
@@ -1,34 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.deploy;
-
-import org.I0Itec.zkclient.ZkClient;
-
-public class A {
-
- public A(ZkClient zkClient) {
- try {
- zkClient.createEphemeral(AppConstants.STARTED_ZNODE_2, null);
- } catch (Exception e) {
- System.exit(-1);
- }
-
- }
-
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/AppConstants.java
----------------------------------------------------------------------
diff --git a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/AppConstants.java b/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/AppConstants.java
deleted file mode 100644
index b727f1e..0000000
--- a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/AppConstants.java
+++ /dev/null
@@ -1,28 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.deploy;
-
-public class AppConstants {
- public static final String STARTED_ZNODE_1 = "/s4-test/test-app-1-started";
- public static final String INITIALIZED_ZNODE_1 = "/s4-test/test-app-1-initialized";
-
- public static final String STARTED_ZNODE_2 = "/s4-test/test-app-2-started";
- public static final String INITIALIZED_ZNODE_2 = "/s4-test/test-app-2-initialized";
-
-}
http://git-wip-us.apache.org/repos/asf/incubator-s4/blob/96eddf87/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/TestApp.java
----------------------------------------------------------------------
diff --git a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/TestApp.java b/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/TestApp.java
deleted file mode 100644
index 3d3fb61..0000000
--- a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/TestApp.java
+++ /dev/null
@@ -1,56 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.s4.deploy;
-
-import org.I0Itec.zkclient.ZkClient;
-import org.apache.s4.core.App;
-import org.apache.zookeeper.CreateMode;
-
-public class TestApp extends App {
-
- private ZkClient zkClient;
-
- @Override
- protected void onClose() {
- // TODO Auto-generated method stub
-
- }
-
- @Override
- protected void onInit() {
- try {
- zkClient = new ZkClient("localhost:" + 2181);
- if (!zkClient.exists("/s4-test")) {
- zkClient.create("/s4-test", null, CreateMode.PERSISTENT);
- }
- zkClient.createEphemeral(AppConstants.INITIALIZED_ZNODE_2, null);
- } catch (Exception e) {
- System.exit(-1);
- }
- }
-
- @Override
- protected void onStart() {
- try {
- Class.forName("org.apache.s4.deploy.A").getConstructor(ZkClient.class).newInstance(zkClient);
- } catch (Exception e) {
- System.exit(-1);
- }
- }
-}