[GitHub] tinkerpop pull request #466: TINKERPOP-1525 Avoid starting VP worker iterati...

[dalaro <gi...@git.apache.org>]

GitHub user dalaro opened a pull request:

    https://github.com/apache/tinkerpop/pull/466

    TINKERPOP-1525 Avoid starting VP worker iterations that never end (Spark 2.0 version)

    This is exactly like #462, except that it tracks a change except it tracks a switch between Spark 1.6 and 2.0 away from functions that manipulate iterables to those that manipulate iterators.
    
    Assuming #462 eventually gets into master, and assuming that TINKERPOP-1389 eventually merges with master, the second merge will conflict.  It still seems marginally safer to make this change in parallel on TINKERPOP-1389 and master/tp32 than just the latter, since the conflict will look more like "oh i better keep one of these two almost-identical edge-case checks" than "oh the Spark 1.x branch had some silly edge case check that I can just delete for 2.0".

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dalaro/incubator-tinkerpop TINKERPOP-1525-for-TINKERPOP-1389

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/tinkerpop/pull/466.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #466
    
----
commit 2a8f741190beebd7b7e6a9ff7922afb9b6807fa5
Author: Dan LaRocque <da...@hopcount.org>
Date:   2016-10-26T00:37:17Z

    Avoid starting VP worker iterations that never end
    
    SparkExecutor.executeVertexProgramIteration was written in such a way
    that an empty RDD partition would cause it to invoke
    VertexProgram.workerIterationStart without ever invoking
    VertexProgram.workerIterationEnd.  This seems like a contract
    violation.  I have at least one VP that relies on
    workerIterationStart|End to allocate and release resources.  Failing
    to invoke End like this causes a leak in that VP, as it would for any
    VP that uses that resource management pattern.
    
    (cherry picked from commit 36e1159a80f539b8bd4a884e5c1cf304ec52c4f9;
     this is the same change, except it tracks a switch between Spark 1.6
     and 2.0 away from functions that manipulate iterables to those that
     manipulate iterators)

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop pull request #466: TINKERPOP-1525 Avoid starting VP worker iterati...

[asfgit <gi...@git.apache.org>]

Github user asfgit closed the pull request at:

    https://github.com/apache/tinkerpop/pull/466


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---