You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by rnowling <gi...@git.apache.org> on 2014/08/27 23:12:02 UTC

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changed

GitHub user rnowling opened a pull request:

    https://github.com/apache/spark/pull/2168

    [SPARK-3263][GraphX] Fix changed

    PR #720 made multiple changes to GraphGenerator.logNormalGraph including:
    
    * Replacing the call to functions for generating random vertices and edges with in-line implementations with different equations. Based on reading the Pregel paper, I believe the in-line functions are incorrect.
    * Hard-coding of RNG seeds so that method now generates the same graph for a given number of vertices, edges, mu, and sigma -- user is not able to override seed or specify that seed should be randomly generated. 
    * Backwards-incompatible change to logNormalGraph signature with introduction of new required parameter.
    * Failed to update scala docs and programming guide for API changes
    * Added a Synthetic Benchmark in the examples.
    
    This PR:
    * Removes the in-line calls and calls original vertex / edge generation functions again
    * Adds an optional seed parameter for deterministic behavior (when desired)
    * Keeps the number of partitions parameter that was added.
    * Keeps compatibility with the synthetic benchmark example

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rnowling/spark graphgenrand

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/2168.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2168
    
----
commit 015010c4213594be5c9086f82743adebfd88a936
Author: RJ Nowling <rn...@gmail.com>
Date:   2014-08-26T20:17:59Z

    Fixed GraphGenerator logNormalGraph API to make backward-incompatible change in commit 894ecde04

commit c1831368ce36bbac48c2733ffa3077be453aedae
Author: RJ Nowling <rn...@gmail.com>
Date:   2014-08-27T20:03:45Z

    Fix to deterministic GraphGenerators.logNormalGraph that allows generating graphs randomly using optional seed.

commit 684804d4c5c33bd4d50a179b15e3b1b0375ff1d8
Author: RJ Nowling <rn...@gmail.com>
Date:   2014-08-27T21:06:51Z

    revert PR #720 which introduce errors in logNormalGraph and messed up seeding of RNGs.  Add user-defined optional seed for deterministic behavior

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53730896
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19404/consoleFull) for   PR 2168 at commit [`799f002`](https://github.com/apache/spark/commit/799f00203dc30f070f1725bca697977a54b03b84).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53642025
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19348/consoleFull) for   PR 2168 at commit [`684804d`](https://github.com/apache/spark/commit/684804d4c5c33bd4d50a179b15e3b1b0375ff1d8).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53731054
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19403/consoleFull) for   PR 2168 at commit [`c70868d`](https://github.com/apache/spark/commit/c70868dd991f375c73015587fc48f0139e7bc52f).
     * This patch **fails** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53757353
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19408/consoleFull) for   PR 2168 at commit [`785ac70`](https://github.com/apache/spark/commit/785ac7026a65e49786c955fd8a4ccc2ed89c1fbf).
     * This patch **fails** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53867935
  
    It seems 3 tests failed but they were unrelated to my changes. The
    GraphGeneratorsSuite I added passed without issue.
    
    Are other PRs having similar issues?
    
    On Friday, August 29, 2014, Apache Spark QA <no...@github.com>
    wrote:
    
    > QA tests have finished
    > <https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19450/consoleFull>
    > for PR 2168 at commit e11918e
    > <https://github.com/apache/spark/commit/e11918e257be8526867da302c61f4b973b32e41f>
    > .
    >
    >    - This patch *fails* unit tests.
    >    - This patch merges cleanly.
    >    - This patch adds no public classes.
    >
    > —
    > Reply to this email directly or view it on GitHub
    > <https://github.com/apache/spark/pull/2168#issuecomment-53842447>.
    >
    
    
    -- 
    em rnowling@gmail.com
    c 954.496.2314


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53731050
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19405/consoleFull) for   PR 2168 at commit [`43949ad`](https://github.com/apache/spark/commit/43949ad1c06823ed6385d64ff40810d370d28e70).
     * This patch **fails** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53746828
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19408/consoleFull) for   PR 2168 at commit [`785ac70`](https://github.com/apache/spark/commit/785ac7026a65e49786c955fd8a4ccc2ed89c1fbf).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53730908
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19403/consoleFull) for   PR 2168 at commit [`c70868d`](https://github.com/apache/spark/commit/c70868dd991f375c73015587fc48f0139e7bc52f).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/2168


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53761979
  
    Thanks, @jegonzal !


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53722074
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19402/consoleFull) for   PR 2168 at commit [`1c8fc44`](https://github.com/apache/spark/commit/1c8fc4407cabc8c592bde4b696bd09f5b3766da5).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53731069
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19404/consoleFull) for   PR 2168 at commit [`799f002`](https://github.com/apache/spark/commit/799f00203dc30f070f1725bca697977a54b03b84).
     * This patch **fails** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53761843
  
    I fixed comparison of Edge objects.  I hadn't been using testOnly properly before so I didn't catch this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53729660
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19401/consoleFull) for   PR 2168 at commit [`dfbb6dd`](https://github.com/apache/spark/commit/dfbb6dd86256ba176a304d103c9b0b9fe00d302b).
     * This patch **passes** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53839883
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19450/consoleFull) for   PR 2168 at commit [`e11918e`](https://github.com/apache/spark/commit/e11918e257be8526867da302c61f4b973b32e41f).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53730337
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19402/consoleFull) for   PR 2168 at commit [`1c8fc44`](https://github.com/apache/spark/commit/1c8fc4407cabc8c592bde4b696bd09f5b3766da5).
     * This patch **passes** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53727924
  
    thanks @ankurdave .  I'd also like to recommend @srowen as a reviewer since he made a few contributions to the GraphGenerator in the past.
    
    I've made the following changes based on your feedback:
    * Added documentation for logNormalGraph parameters
    * Add optional seed parameter to SynthBenchmark with a default of generating random seeds (more below)
    
    I also found and fixed the following bugs:
    * Fixed a bug in generateRandomEdges where the number of edges produced was not the out-degree but the total number of vertices -- now produces the correct number of edges.
    * Updated SynthBenchmark docs to correct nverts parameter name
    * Fixed failure to call count() before printing result of connected components in SynthBenchmark
    
    and I started unit tests for GraphGenerators -- only covers the logNormalGraph, generateRandomEdges, and sampleLogNormal functions for now but it's a start.
    
    I understand the concern about seeding and benchmarks.  However, I also think the use of optional parameters for seeds can be subtle and problematic.  Most random functions (RNGs, etc.) generate a random seed when one is not given, so users expect that.  I'm often tripped up when the default is a fixed seed.  Since users may use logNormalGraph for statistical testing and other work, I would highly prefer that the default is a random seed.
    
    As for SynthBenchmark, I followed the same principle of least surprise.  When no seed is given, one is randomly generated.  If you feel the seed should be hard-coded in the benchmark by default, however, I'd be happy to change that.
    
    At this stage, I don't want to introduce backwards-incompatible API changes unless we know the new API will be there for some time.  When a new API is introduced, the seed parameter should be required (rather than optional) to remove the ambiguity and subtly of the issue.
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53762525
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19413/consoleFull) for   PR 2168 at commit [`e11918e`](https://github.com/apache/spark/commit/e11918e257be8526867da302c61f4b973b32e41f).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53721328
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19401/consoleFull) for   PR 2168 at commit [`dfbb6dd`](https://github.com/apache/spark/commit/dfbb6dd86256ba176a304d103c9b0b9fe00d302b).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by jegonzal <gi...@git.apache.org>.
Github user jegonzal commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53760885
  
    The code changes look good to me (and were badly need).  Thanks for fixing it!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by ankurdave <gi...@git.apache.org>.
Github user ankurdave commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53662064
  
    @rnowling Thanks for this change! I merged #720 hastily, so it's good to fix any problems it introduced. A few comments:
    
    1. SynthBenchmark is most useful if it's deterministic, so what do you think about making `seed` default to 0 instead of -1, or changing SynthBenchmark to set the seed to 0? That way the generated graph should by default be identical to what it was before, aside from the log-normal formula change.
    2. It would be good to document the parameters to logNormalGraph briefly.
    3. Eventually we should probably switch from Scala optional parameters to the binary-compatible (but much more verbose) builder pattern as in MLlib. For now I think optional parameters are fine, though.
    
    @jegonzal Could you take a look at the formula change?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53772862
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19413/consoleFull) for   PR 2168 at commit [`e11918e`](https://github.com/apache/spark/commit/e11918e257be8526867da302c61f4b973b32e41f).
     * This patch **fails** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53730898
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19405/consoleFull) for   PR 2168 at commit [`43949ad`](https://github.com/apache/spark/commit/43949ad1c06823ed6385d64ff40810d370d28e70).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53642189
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19348/consoleFull) for   PR 2168 at commit [`684804d`](https://github.com/apache/spark/commit/684804d4c5c33bd4d50a179b15e3b1b0375ff1d8).
     * This patch **fails** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/2168#discussion_r16842842
  
    --- Diff: graphx/src/main/scala/org/apache/spark/graphx/util/GraphGenerators.scala ---
    @@ -48,33 +48,36 @@ object GraphGenerators {
        * Ilan Horn, Naty Leiser, and Grzegorz Czajkowski. 2010.
        * Pregel: a system for large-scale graph processing. SIGMOD '10.
        *
    -   * @param sc
    -   * @param numVertices
    -   * @param mu
    -   * @param sigma
    -   * @return
    +   * If the seed is set to 0 (default), the seeds are set deterministically
    --- End diff --
    
    @rnowling Oh I don't think I know much of anything about GraphX -- the one PR I submitted was a minor math issue. I looked anyway and the only comment I have is that I don't see handling for the case of seed == 0 in the code, but the scaladoc suggests this does something special.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53774037
  
    These test failures are occurring in PySpark and seem unrelated.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-54355591
  
    Hooray! :) Thanks @ankurdave  and @JoshRosen 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by ankurdave <gi...@git.apache.org>.
Github user ankurdave commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-54364272
  
    Great! Merging into master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53729304
  
    Thanks, @srowen !  I forgot to update the docs as I was making changes. :(  Fixed now!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-54345284
  
    Jenkins, retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-54345882
  
      [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19677/consoleFull) for   PR 2168 at commit [`e11918e`](https://github.com/apache/spark/commit/e11918e257be8526867da302c61f4b973b32e41f).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53842447
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19450/consoleFull) for   PR 2168 at commit [`e11918e`](https://github.com/apache/spark/commit/e11918e257be8526867da302c61f4b973b32e41f).
     * This patch **fails** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-54355293
  
      [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19677/consoleFull) for   PR 2168 at commit [`e11918e`](https://github.com/apache/spark/commit/e11918e257be8526867da302c61f4b973b32e41f).
     * This patch **passes** unit tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by rnowling <gi...@git.apache.org>.
Github user rnowling commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-54306706
  
    Thanks @ankurdave   Tests didn't seem to run, though...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by ankurdave <gi...@git.apache.org>.
Github user ankurdave commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-54243007
  
    Might as well try again: Jenkins, retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:

    https://github.com/apache/spark/pull/2168#issuecomment-53839706
  
    Jenkins, retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org