You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "yuhao yang (JIRA)" <ji...@apache.org> on 2015/05/11 08:42:00 UTC
[jira] [Comment Edited] (SPARK-7514) Add MinMaxScaler to feature
transformation
[ https://issues.apache.org/jira/browse/SPARK-7514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14537651#comment-14537651 ]
yuhao yang edited comment on SPARK-7514 at 5/11/15 6:41 AM:
------------------------------------------------------------
Thanks Joseph, just one concern for using center as it will change the core function from
Normalized( x ) = (x - min) / (max - min) * scale + newBase
to
Normalized( x ) = ((x - min) / (max - min) - 0.5 )* scale + center
which seems not as straightforward.
Sure we can further discuss it over code.
was (Author: yuhaoyan):
Thanks Joseph, just one concern for using center as it will change the core function from
Normalized( x ) = (x - min) / (max - min) * scale + newBase
to
Normalized( x ) = ((x - min) / (max - min) - 0.5 )* scale + center
which seems be to not as straightforward.
Sure we can further discuss it over code.
> Add MinMaxScaler to feature transformation
> ------------------------------------------
>
> Key: SPARK-7514
> URL: https://issues.apache.org/jira/browse/SPARK-7514
> Project: Spark
> Issue Type: New Feature
> Components: MLlib
> Reporter: yuhao yang
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Add a popular scaling method to feature component, which is commonly known as min-max normalization or Rescaling.
> Core function is,
> Normalized( x ) = (x - min) / (max - min) * scale + newBase
> where newBase and scale are parameters of the VectorTransformer. newBase is the new minimum number for the feature, and scale controls the range after transformation. This is a little complicated than the basic MinMax normalization, yet it provides flexibility so that users can control the range more specifically. like [0.1, 0.9] in some NN application.
> for case that max == min, 0.5 is used as the raw value.
> reference:
> http://en.wikipedia.org/wiki/Feature_scaling
> http://stn.spotfire.com/spotfire_client_help/index.htm#norm/norm_scale_between_0_and_1.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org