You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by srowen <gi...@git.apache.org> on 2014/12/30 13:47:19 UTC
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/3842
SPARK-2757 [BUILD] [STREAMING] Add Mima test for Spark Sink after 1.10 is released
Re-enable MiMa for Streaming Flume Sink module, now that 1.1.0 is released, per the JIRA TO-DO. That's pretty much all there is to this.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srowen/spark SPARK-2757
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3842.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3842
----
commit 0e5ba5cefaca04c188aadf5309ca6d5dffe1c63f
Author: Sean Owen <so...@cloudera.com>
Date: 2014-12-30T12:46:10Z
Re-enable MiMa for Streaming Flume Sink module
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68369026
[Test build #24895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24895/consoleFull) for PR 3842 at commit [`50ff80e`](https://github.com/apache/spark/commit/50ff80e4498c2cb0a30793fb41fa2d20942811d6).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68417257
A quick pass shows me that `spark-streaming-flume` uses the following from the `spark-streaming-flume-sink` module
- The generated Avro protocol classes - SparkFlumeProtcol, EventBatch, SparkSinkEvent
- SparkSinkUtils class
Should we consider adding all the other sink classes (TransactionProcessor, SparkAvroCallbackHandler, etc.) to Mima excludes right now? Or progressively add them as required?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68359334
[Test build #24893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24893/consoleFull) for PR 3842 at commit [`0e5ba5c`](https://github.com/apache/spark/commit/0e5ba5cefaca04c188aadf5309ca6d5dffe1c63f).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68369035
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24895/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68429122
I believe we need to track only the Avro classes and the SparkSink class
(not sure if we need binary compat for this either - I think API compat is
all we need for even the SparkSink class). Other than that we should be
fine since the jar that contains the sink itself has the other classes, so
binary compat isn't an issue.
On Tuesday, December 30, 2014, Sean Owen <no...@github.com> wrote:
> @harishreedharan <https://github.com/harishreedharan> might know better;
> it was his suggestion to track this with MiMa. I suppose it can be enabled,
> to err on the side of tracking these things, and exclude/disable later when
> needed?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/3842#issuecomment-68426102>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68354029
[Test build #24893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24893/consoleFull) for PR 3842 at commit [`0e5ba5c`](https://github.com/apache/spark/commit/0e5ba5cefaca04c188aadf5309ca6d5dffe1c63f).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68426102
@harishreedharan might know better; it was his suggestion to track this with MiMa. I suppose it can be enabled, to err on the side of tracking these things, and exclude/disable later when needed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68360108
Interesting situation. There is a MiMa failure since SPARK-3154 / https://github.com/apache/spark/commit/bcb5cdad614d4fce43725dfec3ce88172d2f8c11 changed a method after 1.2.0, but, it's `private[sink]`. I should just exclude this failure I believe.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3842
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68417061
Just wondering what is necessity for binary compatibility here? I am guessing that one should be able to operation Spark Streaming 1.2 programs with a deployed Sink from 1.0, right? For that what is API that needs to stay compatible? Isnt it only the Avro protocol? So we should figure out what are all the stuff that are needed make MIMA ignore the rest, isnt it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68361334
[Test build #24895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24895/consoleFull) for PR 3842 at commit [`50ff80e`](https://github.com/apache/spark/commit/50ff80e4498c2cb0a30793fb41fa2d20942811d6).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68387718
+1. Looks good to me!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68477169
Okay @srowen lets do that - err on the side of tracking these things and disable later when needed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2757 [BUILD] [STREAMING] Add Mima test f...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3842#issuecomment-68359340
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24893/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org