mlib model viewing and saving

[Sameer Tilak <ss...@live.com>]

Hi All,
I have a mlib model:
val model = DecisionTree.train(parsedData, Regression, Variance, maxDepth)

I see model has following methods:algo           asInstanceOf   isInstanceOf   predict        toString       topNode
model.topNode outputs:org.apache.spark.mllib.tree.model.Node = id = 0, isLeaf = false, predict = 0.5, split = Some(Feature = 87, threshold = 0.7931471805599453, featureType =  Continuous, categories = List()), stats = Some(gain = 0.893333, impurity = 0.350000, left impurity = 0.122222, right impurity = 0.000000, predict = 0.500000)
I was wondering what is the best way to look at the model. We want to see what the decision tree looks like-- which features are selected, the details of splitting, what is the depth etc. Is there an easy way to see that? I can traverse it recursively using topNode.leftNode and topNode.rightNode. However, was wondering if there is any way to look at the model and also to save it on the hdfs for later use. 
 		 	   		  

Re: mlib model viewing and saving

[Joseph Bradley <jo...@databricks.com>]

Currently, printing (toString) gives a human-readable version of the tree,
but it is not a format which is easy to save and load.  That sort of
serialization is in the works, but not available for trees right now.
 (Note that the current master actually has toString (for a short summary
of the tree) and toDebugString (for a full printout of the model).
Joseph

On Fri, Aug 15, 2014 at 4:28 PM, Sameer Tilak <ss...@live.com> wrote:

> Hi All,
>
> I have a mlib model:
>
> val model = DecisionTree.train(parsedData, Regression, Variance, maxDepth)
>
>
> I see model has following methods:
> algo           asInstanceOf   isInstanceOf   predict        toString
> topNode
>
> model.topNode outputs:
> org.apache.spark.mllib.tree.model.Node = id = 0, isLeaf = false, predict =
> 0.5, split = Some(Feature = 87, threshold = 0.7931471805599453, featureType
> =  Continuous, categories = List()), stats = Some(gain = 0.893333, impurity
> = 0.350000, left impurity = 0.122222, right impurity = 0.000000, predict =
> 0.500000)
>
> I was wondering what is the best way to look at the model. We want to see
> what the decision tree looks like-- which features are selected, the
> details of splitting, what is the depth etc. Is there an easy way to see
> that? I can traverse it recursively using topNode.leftNode and topNode.rightNode.
> However, was wondering if there is any way to look at the model and also to
> save it on the hdfs for later use.
>
>