You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Glenn Weidner (JIRA)" <ji...@apache.org> on 2015/05/13 02:41:00 UTC
[jira] [Comment Edited] (SPARK-7425) spark.ml Predictor should
support other numeric types for label
[ https://issues.apache.org/jira/browse/SPARK-7425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541099#comment-14541099 ]
Glenn Weidner edited comment on SPARK-7425 at 5/13/15 12:40 AM:
----------------------------------------------------------------
Thank you for the tip!
In PredictorParams.validateAndTransformSchema, I've replaced SchemaUtils.checkColumnType with isInstanceOf[NumericType] as shown below
(where NumericType imported from org.apache.spark.sql.types.NumericType).
if (fitting)
// TODO: Allow other numeric types
//SchemaUtils.checkColumnType(schema, $(labelCol), DoubleType)
val actualDataType = schema($(labelCol)).dataType
require(actualDataType.isInstanceOf[NumericType],
s"Column $labelCol must be of type NumericType but was actually $actualDataType.")
A quick debug run of existing LinearRegressionSuite stepped into modified PredictorParams.validateAndTransformSchema and verified test still passed with DoubleType. Next step is to try test with other numeric types.
was (Author: gweidner):
Thank you for the tip!
In PredictorParams.validateAndTransformSchema, I've replaced SchemaUtils.checkColumnType with isInstanceOf[NumericType] as shown below
(where NumericType imported from org.apache.spark.sql.types.NumericType).
if (fitting) {
// TODO: Allow other numeric types
//SchemaUtils.checkColumnType(schema, $(labelCol), DoubleType)
val actualDataType = schema($(labelCol)).dataType
require(actualDataType.isInstanceOf[NumericType],
s"Column $labelCol must be of type NumericType but was actually $actualDataType.")
}
A quick debug run of existing LinearRegressionSuite stepped into modified PredictorParams.validateAndTransformSchema and verified test still passed with DoubleType. Next step is to try test with other numeric types.
> spark.ml Predictor should support other numeric types for label
> ---------------------------------------------------------------
>
> Key: SPARK-7425
> URL: https://issues.apache.org/jira/browse/SPARK-7425
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Joseph K. Bradley
> Priority: Minor
> Labels: starter
>
> Currently, the Predictor abstraction expects the input labelCol type to be DoubleType, but we should support other numeric types. This will involve updating the PredictorParams.validateAndTransformSchema method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org