[jira] [Created] (TINKERPOP-2424) Reduce chance for OOME with large results to Java driver

["Stephen Mallette (Jira)" <ji...@apache.org>]

Stephen Mallette created TINKERPOP-2424:
-------------------------------------------

             Summary: Reduce chance for OOME with large results to Java driver
                 Key: TINKERPOP-2424
                 URL: https://issues.apache.org/jira/browse/TINKERPOP-2424
             Project: TinkerPop
          Issue Type: Improvement
          Components: driver
    Affects Versions: 3.4.8
            Reporter: Stephen Mallette


Originally mentioned here:

https://groups.google.com/g/gremlin-users/c/I4HQC9JkzSo/m/fYfd5o0UAQAJ

and pretty easy to create with an empty TinkerGraph in Gremlin Server with {{evaluationTimeout}} set to something large using this script in the Gremlin Console with {{-Xmx512}}:

{code}
cluster = Cluster.open()
client = cluster.connect()
client.submit("g.addV().as('a').addE('self').iterate()")
rs = client.submit("g.V().emit().repeat(out()).valueMap(true).limit(10000000)");[]
iterator = rs.iterator();[]
x = 0
while(iterator.hasNext()) {
  x++
  if (x % 10000 == 0) {
    System.out.println(x + "-[" + rs.getAvailableItemCount() + "]-"+ iterator.next());
  }
}
{code}

The {{LinkedBlockingQueue}} of the {{ResultQueue}} is unbounded and can fill faster than can be consumed and on a system with limited memory an OOME can loom. 

While we tend to discourage iteration of large result sets {{e.g. g.V()}} I suppose we should do what we can to keep users out of OOME situations if we can. Not sure of the best way to do this but some simple experimentation showed that bounding the queue helps (tried with 100000) but does require the adding of new results to be blocked until more are consumed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)