[jira] [Commented] (MAHOUT-874) Extract Writables into a separate module to allow smaller dependencies

["Ted Dunning (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAHOUT-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144474#comment-13144474 ] 

Ted Dunning commented on MAHOUT-874:
------------------------------------

With a quick slash, the dependencies are down to this
{code}
org.apache.mahout:mahout-math:0.6-SNAPSHOT
org.apache.hadoop:hadoop-core:0.20.204.0
commons-cli:commons-cli:1.2
commons-httpclient:commons-httpclient:3.0.1
commons-codec:commons-codec:1.4
commons-configuration:commons-configuration:1.6
org.codehaus.jackson:jackson-core-asl:1.8.2
org.codehaus.jackson:jackson-mapper-asl:1.8.2
org.slf4j:slf4j-api:1.6.1
org.slf4j:slf4j-jcl:1.6.1
junit:junit:4.8.2
{code}

The number of classes I had to move was a bit surprising.  THis is will result in some ugliness in coding because different pieces of packages will need to be in different places.

The complete list of classes in the writable jar is this:

{code}
src/main/java/org/apache/mahout/cf/taste/hadoop/EntityCountWritable.java
src/main/java/org/apache/mahout/cf/taste/hadoop/EntityEntityWritable.java
src/main/java/org/apache/mahout/cf/taste/hadoop/EntityPrefWritable.java
src/main/java/org/apache/mahout/cf/taste/hadoop/EntityPrefWritableArrayWritable.java
src/main/java/org/apache/mahout/cf/taste/hadoop/item/PrefAndSimilarityColumnWritable.java
src/main/java/org/apache/mahout/cf/taste/hadoop/item/VectorAndPrefsWritable.java
src/main/java/org/apache/mahout/cf/taste/hadoop/item/VectorOrPrefWritable.java
src/main/java/org/apache/mahout/cf/taste/hadoop/RecommendedItemsWritable.java
src/main/java/org/apache/mahout/cf/taste/impl/recommender/GenericRecommendedItem.java
src/main/java/org/apache/mahout/cf/taste/recommender/RecommendedItem.java
src/main/java/org/apache/mahout/classifier/sgd/PolymorphicWritable.java
src/main/java/org/apache/mahout/clustering/AbstractCluster.java
src/main/java/org/apache/mahout/clustering/Cluster.java
src/main/java/org/apache/mahout/clustering/ClusterObservations.java
src/main/java/org/apache/mahout/clustering/Model.java
src/main/java/org/apache/mahout/clustering/spectral/common/IntDoublePairWritable.java
src/main/java/org/apache/mahout/clustering/spectral/common/VertexWritable.java
src/main/java/org/apache/mahout/clustering/WeightedPropertyVectorWritable.java
src/main/java/org/apache/mahout/clustering/WeightedVectorWritable.java
src/main/java/org/apache/mahout/common/ClassUtils.java
src/main/java/org/apache/mahout/common/IntPairWritable.java
src/main/java/org/apache/mahout/common/parameters/Parameter.java
src/main/java/org/apache/mahout/common/parameters/Parametered.java
src/main/java/org/apache/mahout/graph/linkanalysis/VectorElementWritable.java
src/main/java/org/apache/mahout/math/hadoop/stochasticsvd/DenseBlockWritable.java
src/main/java/org/apache/mahout/math/hadoop/stochasticsvd/SparseRowBlockWritable.java
src/main/java/org/apache/mahout/math/hadoop/stochasticsvd/SplitPartitionedWritable.java
src/main/java/org/apache/mahout/math/MatrixWritable.java
src/main/java/org/apache/mahout/math/MultiLabelVectorWritable.java
src/main/java/org/apache/mahout/math/Varint.java
src/main/java/org/apache/mahout/math/VarIntWritable.java
src/main/java/org/apache/mahout/math/VarLongWritable.java
src/main/java/org/apache/mahout/math/VectorWritable.java
{code}

The disappointment is really with the Cluster class.  It had to move and that pulled a bunch of other things across.

What is the sense about this?

                
> Extract Writables into a separate module to allow smaller dependencies
> ----------------------------------------------------------------------
>
>                 Key: MAHOUT-874
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-874
>             Project: Mahout
>          Issue Type: Improvement
>            Reporter: Ted Dunning
>
> The theory is that we can have a smaller jar if we only include writable classes and their exact dependencies.
> I have a prototype, but it has some funky characteristics which I would like to discuss.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira