[jira] [Commented] (TINKERPOP-1525) Plug VertexProgram iteration leak on empty Spark RDD partitions

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TINKERPOP-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603693#comment-15603693 ] 

ASF GitHub Bot commented on TINKERPOP-1525:
-------------------------------------------

GitHub user dalaro opened a pull request:

    https://github.com/apache/tinkerpop/pull/462

    TINKERPOP-1525 Avoid starting VP worker iterations that never end

    SparkExecutor.executeVertexProgramIteration was written in such a way
    that an empty RDD partition would cause it to invoke
    VertexProgram.workerIterationStart without ever invoking
    VertexProgram.workerIterationEnd.  This seems like a contract
    violation.  I have at least one VP that relies on
    workerIterationStart|End to allocate and release resources.  Failing
    to invoke End like this causes a leak in that VP, as it would for any
    VP that uses that resource management pattern.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dalaro/incubator-tinkerpop TINKERPOP-1525-tp32

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/tinkerpop/pull/462.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #462
    
----
commit 1fc6ec5d7c20d6163c61f84198e3e68457e23eae
Author: Dan LaRocque <da...@hopcount.org>
Date:   2016-10-21T21:04:30Z

    Avoid starting VP worker iterations that never end
    
    SparkExecutor.executeVertexProgramIteration was written in such a way
    that an empty RDD partition would cause it to invoke
    VertexProgram.workerIterationStart without ever invoking
    VertexProgram.workerIterationEnd.  This seems like a contract
    violation.  I have at least one VP that relies on
    workerIterationStart|End to allocate and release resources.  Failing
    to invoke End like this causes a leak in that VP, as it would for any
    VP that uses that resource management pattern.

----


> Plug VertexProgram iteration leak on empty Spark RDD partitions
> ---------------------------------------------------------------
>
>                 Key: TINKERPOP-1525
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-1525
>             Project: TinkerPop
>          Issue Type: Bug
>    Affects Versions: 3.2.3, 3.3.0
>            Reporter: Dan LaRocque
>
> If SparkExecutor gets an RDD with empty partitions, it can invoke {{VertexProgram.workerIterationStart}} without ever invoking {{VertexProgram.workerIterationEnd}}.
> For vertex programs that allocate and release meaningful resources in the start/end methods, this can lead to resource leaks.
> I already tested a fix that I made against the 3.2 series.  I will submit PRs momentarily.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)