You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Yakov Kerzhner <yk...@hbk.com> on 2021/02/22 23:38:24 UTC

A serious bug in the fitting of a binary logistic regression.

I have written up a JIRA, and there is a gist attached that has code that reproduces the issue.  This is a fairly serious issue as it probably affects everyone who uses spark to fit binary logistic regressions.
https://issues.apache.org/jira/browse/SPARK-34448
Would be great if someone who understands binary logistic regressions and the implementation in scala to take a look.

Re: A serious bug in the fitting of a binary logistic regression.

Posted by Sean Owen <sr...@gmail.com>.

I'll take a look. At a glance - is it converging? might turn down the
tolerance to check.
Also what does scikit learn say on the same data? we can continue on the
JIRA.

On Mon, Feb 22, 2021 at 5:42 PM Yakov Kerzhner <yk...@hbk.com> wrote:

> I have written up a JIRA, and there is a gist attached that has code that
> reproduces the issue.  This is a fairly serious issue as it probably
> affects everyone who uses spark to fit binary logistic regressions.
> https://issues.apache.org/jira/browse/SPARK-34448
> Would be great if someone who understands binary logistic regressions and
> the implementation in scala to take a look.
>