You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/12/22 15:59:00 UTC
[jira] [Commented] (SPARK-14023) Make exceptions consistent
regarding fields and columns
[ https://issues.apache.org/jira/browse/SPARK-14023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16727490#comment-16727490 ]
ASF GitHub Bot commented on SPARK-14023:
----------------------------------------
srowen opened a new pull request #23373: [SPARK-14023][CORE] Don't reference 'field' in StructField errors for clarity in exceptions
URL: https://github.com/apache/spark/pull/23373
## What changes were proposed in this pull request?
Variation of https://github.com/apache/spark/pull/20500
I cheated by not referencing fields or columns at all as this exception propagates in contexts where both would be applicable.
## How was this patch tested?
Existing tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Make exceptions consistent regarding fields and columns
> -------------------------------------------------------
>
> Key: SPARK-14023
> URL: https://issues.apache.org/jira/browse/SPARK-14023
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 2.0.0
> Reporter: Jacek Laskowski
> Priority: Trivial
>
> As you can see below, a column is called a field depending on where an exception is thrown. I think it should be "column" everywhere (since that's what has a type from a schema).
> {code}
> scala> lr
> res32: org.apache.spark.ml.regression.LinearRegression = linReg_d9bfe808e743
> scala> lr.fit(ds)
> java.lang.IllegalArgumentException: Field "features" does not exist.
> at org.apache.spark.sql.types.StructType$$anonfun$apply$1.apply(StructType.scala:214)
> at org.apache.spark.sql.types.StructType$$anonfun$apply$1.apply(StructType.scala:214)
> at scala.collection.MapLike$class.getOrElse(MapLike.scala:128)
> at scala.collection.AbstractMap.getOrElse(Map.scala:59)
> at org.apache.spark.sql.types.StructType.apply(StructType.scala:213)
> at org.apache.spark.ml.util.SchemaUtils$.checkColumnType(SchemaUtils.scala:40)
> at org.apache.spark.ml.PredictorParams$class.validateAndTransformSchema(Predictor.scala:50)
> at org.apache.spark.ml.Predictor.validateAndTransformSchema(Predictor.scala:71)
> at org.apache.spark.ml.Predictor.transformSchema(Predictor.scala:116)
> at org.apache.spark.ml.PipelineStage.transformSchema(Pipeline.scala:67)
> at org.apache.spark.ml.Predictor.fit(Predictor.scala:89)
> ... 51 elided
> scala> lr.fit(ds)
> java.lang.IllegalArgumentException: requirement failed: Column label must be of type DoubleType but was actually StringType.
> at scala.Predef$.require(Predef.scala:219)
> at org.apache.spark.ml.util.SchemaUtils$.checkColumnType(SchemaUtils.scala:42)
> at org.apache.spark.ml.PredictorParams$class.validateAndTransformSchema(Predictor.scala:53)
> at org.apache.spark.ml.Predictor.validateAndTransformSchema(Predictor.scala:71)
> at org.apache.spark.ml.Predictor.transformSchema(Predictor.scala:116)
> at org.apache.spark.ml.PipelineStage.transformSchema(Pipeline.scala:67)
> at org.apache.spark.ml.Predictor.fit(Predictor.scala:89)
> ... 51 elided
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org