You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@atlas.apache.org by Ashutosh Mestry via Review Board <no...@reviews.apache.org> on 2020/04/29 17:51:52 UTC

Review Request 72452: Efficiently Searching for Edges Between Vertices

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/
-----------------------------------------------------------

Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.

Bugs: ATLAS-3762
https://issues.apache.org/jira/browse/ATLAS-3762

Repository: atlas

Description
-------

**Problem Definition**
Please refer to JIRA for details.

**Updates**
- Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : Now uses genuine iterators. This reduces number of elements fetched, since the search is linear.
- New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned from _JanusVertex_. Fetching the count is effecient using stream support.

Diffs
-----

graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 9406e26ff
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java eb0206271
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java fdc9fd0b5
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 2b8227a7e
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java d1c1f1255

Diff: https://reviews.apache.org/r/72452/diff/1/

Testing
-------

**Volume testing**
High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.

Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.

**Pre-commit Build**
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/

Thanks,

Ashutosh Mestry

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.


> On April 29, 2020, 7:57 p.m., Sidharth Mishra wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
> > Lines 780 (patched)
> > <https://reviews.apache.org/r/72452/diff/1/?file=2229439#file2229439line781>
> >
> >     Just thinking the below might be more easy to understand if someone is not aware of why we do both edge count.
> >     
> >            AtlasPerfMetrics.MetricRecorder metric = RequestContext.get().startMetricRecord("getRelationshipEdge");
> >     
> >             AtlasEdge           ret         = null;
> >             long toVertexIncomingEdgesCount = graphHelper.getInComingEdgesByLabelCount(toVertex, relationshipLabel);
> >             
> >             if(toVertexIncomingEdgesCount > 0) {
> >                 long fromVertexOutgoingEdgesCount = graphHelper.getOutGoingEdgesByLabelCount(fromVertex, relationshipLabel);
> >                 
> >                 if(toVertexIncomingEdgesCount < fromVertexOutgoingEdgesCount) {
> >                     Iterator<AtlasEdge> edgesIterator = graphHelper.getIncomingEdgesByLabel(toVertex, relationshipLabel);
> >                     ret = getActiveEdgeFromList(fromVertex.getId(), edgesIterator);
> >                 } else if(fromVertexOutgoingEdgesCount > 0) {
> >                     Iterator<AtlasEdge> edgesIterator = graphHelper.getOutGoingEdgesByLabel(fromVertex, relationshipLabel);
> >                     ret = getActiveEdgeFromList(toVertex.getId(), edgesIterator);
> >                 }
> >             }
> >             
> >             RequestContext.get().endMetricRecord(metric);
> >             return ret;

I have changed my approach. Kindly review my latest patch.


- Ashutosh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220553
-----------------------------------------------------------


On April 29, 2020, 5:51 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> -----------------------------------------------------------
> 
> (Updated April 29, 2020, 5:51 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
>     https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : Now uses genuine iterators. This reduces number of elements fetched, since the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> Diffs
> -----
> 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 9406e26ff 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java eb0206271 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 2b8227a7e 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/1/
> 
> 
> Testing
> -------
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Sidharth Mishra <si...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220553
-----------------------------------------------------------




repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
Lines 780 (patched)
<https://reviews.apache.org/r/72452/#comment309045>

    Just thinking the below might be more easy to understand if someone is not aware of why we do both edge count.
    
           AtlasPerfMetrics.MetricRecorder metric = RequestContext.get().startMetricRecord("getRelationshipEdge");
    
            AtlasEdge           ret         = null;
            long toVertexIncomingEdgesCount = graphHelper.getInComingEdgesByLabelCount(toVertex, relationshipLabel);
            
            if(toVertexIncomingEdgesCount > 0) {
                long fromVertexOutgoingEdgesCount = graphHelper.getOutGoingEdgesByLabelCount(fromVertex, relationshipLabel);
                
                if(toVertexIncomingEdgesCount < fromVertexOutgoingEdgesCount) {
                    Iterator<AtlasEdge> edgesIterator = graphHelper.getIncomingEdgesByLabel(toVertex, relationshipLabel);
                    ret = getActiveEdgeFromList(fromVertex.getId(), edgesIterator);
                } else if(fromVertexOutgoingEdgesCount > 0) {
                    Iterator<AtlasEdge> edgesIterator = graphHelper.getOutGoingEdgesByLabel(fromVertex, relationshipLabel);
                    ret = getActiveEdgeFromList(toVertex.getId(), edgesIterator);
                }
            }
            
            RequestContext.get().endMetricRecord(metric);
            return ret;


- Sidharth Mishra


On April 29, 2020, 5:51 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> -----------------------------------------------------------
> 
> (Updated April 29, 2020, 5:51 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
>     https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : Now uses genuine iterators. This reduces number of elements fetched, since the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> Diffs
> -----
> 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 9406e26ff 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java eb0206271 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 2b8227a7e 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/1/
> 
> 
> Testing
> -------
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Madhan Neethiraj <ma...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220550
-----------------------------------------------------------




graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java
Lines 84 (patched)
<https://reviews.apache.org/r/72452/#comment309039>

    edgeLabels => edgeLabel



repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
Lines 272 (patched)
<https://reviews.apache.org/r/72452/#comment309040>

    getAdjacentEdgesByLabelCount() => getAdjacentEdgesCountByLabel()



repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
Lines 455 (patched)
<https://reviews.apache.org/r/72452/#comment309041>

    getOutGoingEdgesByLabelCount() => getOutGoingEdgesCountByLabel()



repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
Lines 459 (patched)
<https://reviews.apache.org/r/72452/#comment309042>

    getInComingEdgesByLabelCount() => getInComingEdgesCountByLabel()



repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
Lines 786 (patched)
<https://reviews.apache.org/r/72452/#comment309043>

    value assigned in #786 is overwritten by #793 or #796 or #799. It looks like the intent is to skip call to graphHelper.getOutGoingEdgesByLabelCount() with incomingEdgeCount is 0. Consider rearragning the code for clarity and readability:
    
      AtlasEdge ret               = null;
      long      incomingEdgeCount = graphHelper.getInComingEdgesByLabelCount(toVertex, relationshipLabel);
    
      if (incomingEdgeCount > 0) {
        long outgoingEdgeCount = graphHelper.getOutGoingEdgesByLabelCount(fromVertex, relationshipLabel);
        
        if (outgoingEdgeCount > 0) {
          if (incomingEdgeCount < outgoingEdgeCount) {
            Iterator<AtlasEdge> edgesIterator = graphHelper.getIncomingEdgesByLabel(toVertex, relationshipLabel);
    
            ret = getActiveEdgeFromList(fromVertex.getId(), edgesIterator);
          } else {
            Iterator<AtlasEdge> edgesIterator = graphHelper.getOutGoingEdgesByLabel(fromVertex, relationshipLabel);
    
            ret = getActiveEdgeFromList(toVertex.getId(), edgesIterator);
          }
        }
      }
      
      return ret;
    
    Also, given we are looking an edge from 'fromVertex' to 'toVertex' - isnt looking for 'incomingEdgeCount > 0' enough? What is the need for 'outgoingEdgeCount'?


- Madhan Neethiraj


On April 29, 2020, 5:51 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> -----------------------------------------------------------
> 
> (Updated April 29, 2020, 5:51 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
>     https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : Now uses genuine iterators. This reduces number of elements fetched, since the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> Diffs
> -----
> 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 9406e26ff 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java eb0206271 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 2b8227a7e 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/1/
> 
> 
> Testing
> -------
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Sidharth Mishra <si...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220557
-----------------------------------------------------------


Ship it!




Ship It!

- Sidharth Mishra


On April 29, 2020, 9:17 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> -----------------------------------------------------------
> 
> (Updated April 29, 2020, 9:17 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
>     https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : Now uses genuine iterators. This reduces number of elements fetched, since the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> Diffs
> -----
> 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 9406e26ff 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java eb0206271 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 2b8227a7e 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/3/
> 
> 
> Testing
> -------
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Sidharth Mishra <si...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220604
-----------------------------------------------------------


Ship it!




Ship It!

- Sidharth Mishra


On April 30, 2020, 4:04 a.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> -----------------------------------------------------------
> 
> (Updated April 30, 2020, 4:04 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
>     https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : Now uses genuine iterators. This reduces number of elements fetched, since the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> **Metrics**
> 
> This difference exists because earlier even _fromVertex_ did not have any edges, the search would iterate through all the edges of the _toVertex_ and return NULL. This condition is detected in the algorithm and used to reduce the search cycle.
> 
> Before: "getRelationshipEdge":{"count":500,"timeTaken":37048}
> After: "getRelationshipEdge":{"count":500,"timeTaken":42}
> 
> 
> Diffs
> -----
> 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 9406e26ff 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java eb0206271 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 2b8227a7e 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/4/
> 
> 
> Testing
> -------
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1860/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.

(Updated April 30, 2020, 4:04 a.m.)

Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.

Changes
-------

Updates include: Updated with latest PC build details.

Bugs: ATLAS-3762
https://issues.apache.org/jira/browse/ATLAS-3762

Repository: atlas

Description
-------

**Problem Definition**
Please refer to JIRA for details.

Diffs
-----

Diff: https://reviews.apache.org/r/72452/diff/4/

Testing (updated)
-------

**Volume testing**
High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.

Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.

**Pre-commit Build**
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1860/

Thanks,

Ashutosh Mestry

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Madhan Neethiraj <ma...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220558
-----------------------------------------------------------


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
Lines 786 (patched)
<https://reviews.apache.org/r/72452/#comment309049>

    Consider removing 'metric1', to avoid any little impact in this heavily used code path.


- Madhan Neethiraj


On April 29, 2020, 9:17 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> -----------------------------------------------------------
> 
> (Updated April 29, 2020, 9:17 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
>     https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : Now uses genuine iterators. This reduces number of elements fetched, since the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> Diffs
> -----
> 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java 9406e26ff 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java eb0206271 
>   graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 2b8227a7e 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/3/
> 
> 
> Testing
> -------
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.

(Updated April 29, 2020, 9:17 p.m.)

Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.

Changes
-------

Updates include:
- Addressed review comments.
- Addressed readability, simplified approach to detect if iterating over edges is needed using the _vertex.hasEdges_ method.

Bugs: ATLAS-3762
https://issues.apache.org/jira/browse/ATLAS-3762

Repository: atlas

Description
-------

**Problem Definition**
Please refer to JIRA for details.

Diffs (updated)
-----

Diff: https://reviews.apache.org/r/72452/diff/2/

Changes: https://reviews.apache.org/r/72452/diff/1-2/

Testing
-------

**Volume testing**
High volume testing makes the edge fetching effcient. Cases where incoming edges was in 1000s and outgoing edges was handful.

Memory footprint has improved since JanusGrpah caches edges and then expires it. Fetching fewer edges will reduce number of items in memory.

**Pre-commit Build**
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/

Thanks,

Ashutosh Mestry