[jira] [Commented] (SPARK-14653) Remove NumericParser and jackson dependency from mllib-local

["Vishnu Prasad (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-14653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15251308#comment-15251308 ] 

Vishnu Prasad commented on SPARK-14653:
---------------------------------------

Hey I've just written an example of a UDT for DenseVector I'm assumming this is what i should do to all the classes in mllib-local that uses NumericParser and jackson.

If what I'm doing is right which package am I supposed to put the UDT's in.

{code:borderStyle=solid}
class DenseVectorUDT extends UserDefinedType[DenseVector] {
  override def sqlType: DataType = ArrayType(DoubleType, containsNull = false)
  override def serialize(features: DenseVector): ArrayData = {
    new GenericArrayData(features.values)
  }
  override def userClass: Class[DenseVector] = classOf[DenseVector]
  override def deserialize(datum: Any): Vector = {
    datum match {
      case jValue: JValue => (jValue \ "type").extract[Int] match {
        case 0 => // sparse
          val size = (jValue \ "size").extract[Int]
          val indices = (jValue \ "indices").extract[Seq[Int]].toArray
          val values = (jValue \ "values").extract[Seq[Double]].toArray
          Vectors.sparse(size, indices, values).toDense
        case 1 => // dense
          val values = (jValue \ "values").extract[Seq[Double]].toArray
          Vectors.dense(values)
        case _ =>
          throw new IllegalArgumentException(s"Cannot parse $json into a vector.")
      }
    }
  }
}
{code}

> Remove NumericParser and jackson dependency from mllib-local
> ------------------------------------------------------------
>
>                 Key: SPARK-14653
>                 URL: https://issues.apache.org/jira/browse/SPARK-14653
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Build, ML
>            Reporter: Xiangrui Meng
>
> After SPARK-14549, we should remove NumericParser and jackson from mllib-local, which were introduced very earlier and now replaced by UDTs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org