[jira] [Commented] (SPARK-3147) Implement A/B testing

["Michael Yannakopoulos (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109402#comment-14109402 ] 

Michael Yannakopoulos commented on SPARK-3147:
----------------------------------------------

Hi Xiangrui,

It would be my pleasure to help in the implementation of this task. Not only it would
enhance my coding skills but it would also help me learn better the theory behind the statistic tests that exist. If you have time and you would like to work together,
I would be glad.

Thanks,
Michael

> Implement A/B testing
> ---------------------
>
>                 Key: SPARK-3147
>                 URL: https://issues.apache.org/jira/browse/SPARK-3147
>             Project: Spark
>          Issue Type: New Feature
>          Components: MLlib, Streaming
>            Reporter: Xiangrui Meng
>
> A/B testing is widely used to compare online models. We can implement A/B testing in MLlib and integrate it with Spark Streaming. For example, we have a PairDStream[String, Double], whose keys are model ids and values are observations (click or not, or revenue associated with the event). With A/B testing, we can tell whether one model is significantly better than another at a certain time. There are some caveats. For example, we should avoid multiple testing and support A/A testing as a sanity check.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org