You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/31 04:30:41 UTC
[GitHub] [spark] zhengruifeng opened a new pull request #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
zhengruifeng opened a new pull request #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078
### What changes were proposed in this pull request?
1, Move the impl of ChiSq from .mllib to the .ml side;
2, in `.mllib.ChiSqTest`, call the impl in `.ml.ChiSquareTest`
### Why are the changes needed?
We should migrate the algs from MLlib to ML
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
existing testsuites
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606417814
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120627/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng opened a new pull request #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
zhengruifeng opened a new pull request #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078
### What changes were proposed in this pull request?
1, Move the impl of ChiSq from .mllib to the .ml side;
2, in `.mllib.ChiSqTest`, call the impl in `.ml.ChiSquareTest`
### Why are the changes needed?
We should migrate the algs from MLlib to ML
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
existing testsuites
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606398596
**[Test build #120627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120627/testReport)** for PR 28078 at commit [`ee2248f`](https://github.com/apache/spark/commit/ee2248f769ad4ee5e7dbac383c2f4f4a63512a76).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606392826
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25328/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606394469
**[Test build #120626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120626/testReport)** for PR 28078 at commit [`9e708d6`](https://github.com/apache/spark/commit/9e708d6f5f12c98c1cf37dea4bd4dea73a6dafb9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607053871
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606392821
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #28078: [SPARK-31309][ML] Migrate
the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607039548
I don't think that means we can't refactor and improve the relationship between .ml and .mllib; I'm just saying I don't think we will deprecate or remove the latter. If this is just moving code, OK, maybe not worth it. If we can reduce duplication, that's OK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606412644
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606398962
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25329/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607053466
**[Test build #120666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120666/testReport)** for PR 28078 at commit [`35c6db2`](https://github.com/apache/spark/commit/35c6db2cd860f1343c447b381243af32afce928e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng closed pull request #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
zhengruifeng closed pull request #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606417809
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606511970
I have a question, @zhengruifeng . According to the new rubric, `mllib` will survive like `HiveContext` forever?
> We should migrate the algs from MLlib to ML
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607078192
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607078196
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120666/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606959114
Thanks, @srowen .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606417460
**[Test build #120627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120627/testReport)** for PR 28078 at commit [`ee2248f`](https://github.com/apache/spark/commit/ee2248f769ad4ee5e7dbac383c2f4f4a63512a76).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606412365
**[Test build #120626 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120626/testReport)** for PR 28078 at commit [`9e708d6`](https://github.com/apache/spark/commit/9e708d6f5f12c98c1cf37dea4bd4dea73a6dafb9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607053876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25365/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606398596
**[Test build #120627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120627/testReport)** for PR 28078 at commit [`ee2248f`](https://github.com/apache/spark/commit/ee2248f769ad4ee5e7dbac383c2f4f4a63512a76).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607077730
**[Test build #120666 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120666/testReport)** for PR 28078 at commit [`35c6db2`](https://github.com/apache/spark/commit/35c6db2cd860f1343c447b381243af32afce928e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606412650
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120626/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606417809
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606398956
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606398956
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #28078: [SPARK-31309][ML] Migrate
the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-611030685
Given the other discussion, I'm not sure we want to change the requests of the chi-squared test method. That doesn't mean there aren't other reasons to do this.
Yes the general idea is to call .mllib from .ml and not the other way around. If that's becoming hard or impossible, that could justify moving the logic.
If there isn't a strong reason I suppose I just wouldn't do it. I don't feel strongly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607078196
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120666/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng closed pull request #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
zhengruifeng closed pull request #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607051386
OK, I did not get the point.
I think it worthwhile to move some impls to the .ml side, since in .ml we can use both Dataset and RDD in implementation. Also beacuse that we will still need to maintain those impls, moving them to .ml may help keeping consistence
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606394469
**[Test build #120626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120626/testReport)** for PR 28078 at commit [`9e708d6`](https://github.com/apache/spark/commit/9e708d6f5f12c98c1cf37dea4bd4dea73a6dafb9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607053871
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606417814
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120627/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #28078: [SPARK-31309][ML] Migrate
the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607210720
It's not a bad change and I don't mind some cleanup personally; some are just formatting changes.
The general pattern is to let .ml reuse code in .mllib where they need to share, rather than both point to each other. I'm sure there are cases of both unfortunately (? I'd have to check). I might not move the code just to move it though unless it buys some simplification.
But yet the logic here doesn't depend on whether .mllib will be removed. That isn't an end goal as far as I can tell.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #28078: [SPARK-31309][ML] Migrate
the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606949347
I think .mllib will be around forever, yes. It's not that crazy; it still provides the RDD API for many of the same operations, and they share mostly one code base under the hood. And RDDs are not going away.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607053876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25365/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607053466
**[Test build #120666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120666/testReport)** for PR 28078 at commit [`35c6db2`](https://github.com/apache/spark/commit/35c6db2cd860f1343c447b381243af32afce928e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606412650
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120626/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606398962
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25329/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606991374
@srowen @dongjoon-hyun OK, we will keep the .mllib, so I will close this PR. Thanks for pointing it out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606392821
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28078:
[SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606412644
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-606392826
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25328/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-610818801
@srowen This PR is mainly moving the code without bringing more simplification. But it maybe the first step to futher improvements, a possible point may be _returning results in rows instead a single row_ (in this PR the methods `chiSquaredDenseFeatures` and `chiSquaredSparseFeatures` are added to directly return rdd of test results, I thought migration is enough to open a PR, so I did not point it out).
For other impls, if I want to make an improvements based on DF or DS, should I just use DF or DS in the .mllib side (since .mllib side are almost impled on RDD) or move it to .ml at first?
> I'm sure there are cases of both unfortunately (? I'd have to check).
Yes, at least GMM was implemented in both side.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-611308132
OK, I will close this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28078: [SPARK-31309][ML]
Migrate the ChiSquareTest from MLlib to ML
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28078: [SPARK-31309][ML] Migrate the ChiSquareTest from MLlib to ML
URL: https://github.com/apache/spark/pull/28078#issuecomment-607078192
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org