You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Vaishali Papneja (Jira)" <ji...@apache.org> on 2020/08/18 15:49:00 UTC
[jira] [Comment Edited] (SPARK-32650) Not able to Deserialize Mleap
bundle with latest spark configuration
[ https://issues.apache.org/jira/browse/SPARK-32650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179711#comment-17179711 ]
Vaishali Papneja edited comment on SPARK-32650 at 8/18/20, 3:48 PM:
--------------------------------------------------------------------
Hi,
Could you have a look at the error message. It says that a particular method named - ImpurityCalculator is not found.
*_java.lang.NoSuchMethodError: org.apache.spark.mllib.tree.impurity.ImpurityCalculator$.getCalculator(Ljava/lang/String;[D)Lorg/apache/spark/mllib/tree/impurity/ImpurityCalculator;_*
Ideally this method should be a part of DecisionTreeClassifier package. I found that this method was previously a part of impurity.scala file.
[https://github.com/apache/spark/pull/17407/files#diff-7a887f33f69f6c1c9989b34e721f2814]
was (Author: vaishalipapneja):
Hi,
Could you have a look at the error message. It says that a particular method named - ImpurityCalculator is not found.
*_java.lang.NoSuchMethodError: org.apache.spark.mllib.tree.impurity.ImpurityCalculator$.getCalculator(Ljava/lang/String;[D)Lorg/apache/spark/mllib/tree/impurity/ImpurityCalculator;_*
Ideally this method should be a part of DecisionTreeClassifier package. Please suggest on this.
> Not able to Deserialize Mleap bundle with latest spark configuration
> --------------------------------------------------------------------
>
> Key: SPARK-32650
> URL: https://issues.apache.org/jira/browse/SPARK-32650
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.0.0
> Reporter: Vaishali Papneja
> Priority: Major
>
> Hi,
>
> I am using Databricks cluster with configuration as Spark: 3.0.0 and Scala: 2.12 and want to create Machine Learning pipelines using MLeap - latest version: 0.16.1.
> In my pipeline, I have created ensemble model. I am to serialize the model to a bundle.
> But the issue comes while deserializing it. Below is the error I am facing:
> _java.lang.NoSuchMethodError: org.apache.spark.mllib.tree.impurity.ImpurityCalculator$.getCalculator(Ljava/lang/String;[D)Lorg/apache/spark/mllib/tree/impurity/ImpurityCalculator;_
>
> I tried using another cluster with the configuration as - spark: 2.4.5, scala: 2.11 and MLeap: 0.16.0. There it is working fine.
> Please suggest.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org