[jira] [Created] (SPARK-14671) Pipeline.setStages needs to handle Array non-covariance

["Joseph K. Bradley (JIRA)" <ji...@apache.org>]

Joseph K. Bradley created SPARK-14671:
-----------------------------------------

             Summary: Pipeline.setStages needs to handle Array non-covariance
                 Key: SPARK-14671
                 URL: https://issues.apache.org/jira/browse/SPARK-14671
             Project: Spark
          Issue Type: Bug
          Components: ML
    Affects Versions: 1.6.1, 2.0.0
            Reporter: Joseph K. Bradley
            Assignee: Joseph K. Bradley
            Priority: Minor


The following code worked for Spark 1.5 but fails for 1.6 (using the {{WritableStage, UnWritableStage}} classes in PipelineSuite):
{code}
val stages0 = Array(new UnWritableStage("b"))
val stages1 = Array(new WritableStage("a"))
val steps = stages0 ++ stages1
val p = new Pipeline().setStages(steps)

p.stages.w(steps)
new ParamPair(p.stages, steps)
{code}

This also occurs with a mix of transformers from ml.feature.  It is because Java Arrays are non-covariant and the addition of MLWritable to some transformers means the {{stages0/1}} arrays above are not of type {{Array[PipelineStage]}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org