You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hugegraph.apache.org by "DanGuge (via GitHub)" <gi...@apache.org> on 2023/07/28 07:07:42 UTC
[GitHub] [incubator-hugegraph] DanGuge opened a new pull request, #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
DanGuge opened a new pull request, #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262
<!--
Thank you very much for contributing to Apache HugeGraph, we are happy that you want to help us improve it!
Here are some tips for you:
1. If this is your first time, please read the [contributing guidelines](https://github.com/apache/hugegraph/blob/master/CONTRIBUTING.md)
2. If a PR fix/close an issue, type the message "close xxx" (xxx is the link of related
issue) in the content, GitHub will auto link it (Required)
3. Name the PR title in "Google Commit Format", start with "feat | fix | perf | refactor | doc | chore",
such like: "feat(core): support the PageRank algorithm" or "fix: wrong break in the compute loop" (module is optional)
skip it if you are unsure about which is the best component.
4. One PR address one issue, better not to mix up multiple issues.
5. Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]` (or click it directly after
published)
-->
## Purpose of the PR
- Add statistics infomation in oltp api response, including the number of vertices and edges traversed at runtime, as well as the time the api takes to execute
- Support full infomation about vertices and edges in oltp api response
<!--
Please explain more context in this section, clarify why the changes are needed.
e.g:
- If you propose a new API, clarify the use case for a new API.
- If you fix a bug, you can clarify why it is a bug, and should be associated with an issue.
-->
## Main Changes
<!-- Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. These change logs are helpful for better ant faster reviews.)
For example:
- If you introduce a new feature, please show detailed design here or add the link of design documentation.
- If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
- If there is a discussion in the mailing list, please add the link. -->
### Statistics information:
* add vertexIterCounter and edgeIterCounter in HugeTraverser.java to track traversed vertices and edges at run time
* modify all oltp apis to add statistics information in response
* modify Serializer.java and JsonSerializer.java to support statistics infomation serialization
### Full information about vertices and edges:
* add 'with_vertex' and 'with_edge' parameter option in oltp apis
* modify oltp apis to support vertex and edge information in api response
* add EdgeRecord in HugeTraverser.java to record edges at run time and generate the edge information returned in api response
* modify Path and PathSet in HugeTraverser.java to support full edge information storage
* modify all traversers to support track of edge information at run time
* modify Serializer.java and JsonSerializer.java to support full infomation serialization about vertices and edges
### Implementation overview
* **N: No Need**
* **✓: Supported**
| | statistics | with_vertex | with_edge |
| ------------------------------- | ---------- | ----------- | --------- |
| Kout Get | **✓** | **N** | **N** |
| Kout Post | **✓** | **✓** | **✓** |
| Kneighbor Get | **✓** | **N** | **N** |
| Kneighbor Post | **✓** | **✓** | **✓** |
| Sameneighbors Get | **✓** | **N** | **N** |
| Sameneighbors Post | **✓** | **✓** | **N** |
| Jaccard Similarity Get | **✓** | **N** | **N** |
| Jaccard Similarity Post | **✓** | **N** | **N** |
| Shortest Path Get | **✓** | **✓** | **✓** |
| All Shortest Paths Get | **✓** | **✓** | **✓** |
| Weighted Shortest Path Get | **✓** | **✓** | **✓** |
| Single Source Shortest Path Get | **✓** | **✓** | **✓** |
| Multi Node Shortest Path Post | **✓** | **✓** | **✓** |
| Paths Get | **✓** | **N** | **N** |
| Paths Post | **✓** | **✓** | **✓** |
| Customized Paths Post | **✓** | **✓** | **✓** |
| Template Paths Post | **✓** | **✓** | **✓** |
| Crosspoints Get | **✓** | **N** | **N** |
| Customized Crosspoints Post | **✓** | **✓** | **✓** |
| Rings Get | **✓** | **✓** | **✓** |
| Rays Get | **✓** | **✓** | **✓** |
| Fusiform Similarity Post | **✓** | **N** | **N** |
## Verifying these changes
<!-- Please pick the proper options below -->
- [ ] Trivial rework / code cleanup without any test coverage. (No Need)
- [ ] Already covered by existing tests, such as *(please modify tests here)*.
- [x] Need tests and can be verified as follows:
- Postman Test: the tests of `Kout Post` and `Single Source Shortest Path Get` are shown below
### Initialize the Graph
* Initialze the graph according to the [link](https://hugegraph.apache.org/docs/clients/restful-api/traverser/#32-detailed-explanation-of-traverser-api-1)
### Kout Post
* Method & Url
```
POST http://localhost:8080/graphs/{graph}/traversers/kout
```
#### with_vertex=false, with_edge=false
* Request Body
```json
{
"source": "1:marko",
"step": {
"direction": "BOTH",
"labels": ["knows", "created"],
"properties": {
"weight": "P.gt(0.1)"
},
"max_degree": 10000,
"skip_degree": 100000
},
"max_depth": 1,
"nearest": true,
"limit": 10000,
"with_vertex": false,
"with_path": true,
"with_edge": false
}
```
* Response Status
```
200
```
* Response Body
```json
{
"kout": [
"1:vadas",
"1:josh",
"2:lop"
],
"size": 3,
"paths": [
{
"objects": [
"1:marko",
"2:lop"
]
},
{
"objects": [
"1:marko",
"1:josh"
]
},
{
"objects": [
"1:marko",
"1:vadas"
]
}
],
"vertices": [
"1:marko",
"1:josh",
"1:vadas",
"2:lop"
],
"edges": [
"S1:marko>1>20160110>S1:vadas",
"S1:marko>2>>S2:lop",
"S1:marko>1>20130220>S1:josh"
],
"measure": {
"edge_iters": 3,
"vertice_iters": 1,
"cost": 72
}
}
```
<img width="1494" alt="image" src="https://github.com/apache/incubator-hugegraph/assets/77946882/2c6e7ca8-afc6-42ef-a6db-d957b2f90879">
#### with_vertex=true, with_edge=true
* Request Body
```json
{
"source": "1:marko",
"step": {
"direction": "BOTH",
"labels": ["knows", "created"],
"properties": {
"weight": "P.gt(0.1)"
},
"max_degree": 10000,
"skip_degree": 100000
},
"max_depth": 1,
"nearest": true,
"limit": 10000,
"with_vertex": true,
"with_path": true,
"with_edge": true
}
```
* Response Status
```
200
```
* Response Body
```json
{
"kneighbor": [
"1:vadas",
"1:josh",
"2:lop",
"1:peter",
"2:ripple"
],
"size": 5,
"paths": [
{
"objects": [
"1:marko",
"2:lop"
]
},
{
"objects": [
"1:marko",
"2:lop",
"1:peter"
]
},
{
"objects": [
"1:marko",
"1:josh"
]
},
{
"objects": [
"1:marko",
"1:vadas"
]
},
{
"objects": [
"1:marko",
"1:josh",
"2:ripple"
]
}
],
"vertices": [
{
"id": "2:ripple",
"label": "software",
"type": "vertex",
"properties": {
"name": "ripple",
"lang": "java",
"price": 199
}
},
{
"id": "1:marko",
"label": "person",
"type": "vertex",
"properties": {
"name": "marko",
"age": 29,
"city": "Beijing"
}
},
{
"id": "1:josh",
"label": "person",
"type": "vertex",
"properties": {
"name": "josh",
"age": 32,
"city": "Beijing"
}
},
{
"id": "1:vadas",
"label": "person",
"type": "vertex",
"properties": {
"name": "vadas",
"age": 27,
"city": "Hongkong"
}
},
{
"id": "1:peter",
"label": "person",
"type": "vertex",
"properties": {
"name": "peter",
"age": 35,
"city": "Shanghai"
}
},
{
"id": "2:lop",
"label": "software",
"type": "vertex",
"properties": {
"name": "lop",
"lang": "java",
"price": 328
}
}
],
"edges": [
{
"id": "S1:josh>2>>S2:ripple",
"label": "created",
"type": "edge",
"outV": "1:josh",
"outVLabel": "person",
"inV": "2:ripple",
"inVLabel": "software",
"properties": {
"weight": 1.0,
"date": "20171210"
}
},
{
"id": "S1:marko>1>20160110>S1:vadas",
"label": "knows",
"type": "edge",
"outV": "1:marko",
"outVLabel": "person",
"inV": "1:vadas",
"inVLabel": "person",
"properties": {
"weight": 0.5,
"date": "20160110"
}
},
{
"id": "S1:marko>2>>S2:lop",
"label": "created",
"type": "edge",
"outV": "1:marko",
"outVLabel": "person",
"inV": "2:lop",
"inVLabel": "software",
"properties": {
"weight": 0.4,
"date": "20171210"
}
},
{
"id": "S1:marko>1>20130220>S1:josh",
"label": "knows",
"type": "edge",
"outV": "1:marko",
"outVLabel": "person",
"inV": "1:josh",
"inVLabel": "person",
"properties": {
"weight": 1.0,
"date": "20130220"
}
},
{
"id": "S1:peter>2>>S2:lop",
"label": "created",
"type": "edge",
"outV": "1:peter",
"outVLabel": "person",
"inV": "2:lop",
"inVLabel": "software",
"properties": {
"weight": 0.2,
"date": "20170324"
}
}
],
"measure": {
"edge_iters": 12,
"vertice_iters": 12,
"cost": 34
}
}
```
<img width="1486" alt="image" src="https://github.com/apache/incubator-hugegraph/assets/77946882/85eec8e9-7e80-450c-9be6-06792dd4133c">
### Single Source Shortest Path Get
* Method & Url
```
http://localhost:8080/graphs/hugegraph/traversers/singlesourceshortestpath?source="1:marko"&with_vertex=false&with_edge=false&weight=weight
```
* Response Status
```
200
```
* Response Body
```json
{
"paths": {
"1:peter": {
"weight": 0.6000000000000001,
"vertices": [
"1:marko",
"2:lop",
"1:peter"
]
},
"2:lop": {
"weight": 0.4,
"vertices": [
"1:marko",
"2:lop"
]
},
"1:josh": {
"weight": 0.8,
"vertices": [
"1:marko",
"2:lop",
"1:josh"
]
},
"2:ripple": {
"weight": 1.8,
"vertices": [
"1:marko",
"2:lop",
"1:josh",
"2:ripple"
]
},
"1:vadas": {
"weight": 0.5,
"vertices": [
"1:marko",
"1:vadas"
]
}
},
"vertices": [
"1:peter",
"2:lop",
"1:josh",
"2:ripple",
"1:marko",
"1:vadas"
],
"edges": [
"S1:josh>2>>S2:ripple",
"S1:marko>1>20160110>S1:vadas",
"S1:josh>2>>S2:lop",
"S1:marko>2>>S2:lop",
"S1:peter>2>>S2:lop"
],
"measure": {
"edge_iters": 12,
"vertice_iters": 6,
"cost": 16
}
}
```
<img width="1483" alt="image" src="https://github.com/apache/incubator-hugegraph/assets/77946882/e49b4dee-b795-4c75-bb37-7b2c104fa1c4">
* Method & Url
```
http://localhost:8080/graphs/hugegraph/traversers/singlesourceshortestpath?source="1:marko"&with_vertex=true&with_edge=true&weight=weight
```
* Response Status
```
200
```
* Response Body
```json
{
"paths": {
"1:peter": {
"weight": 0.6000000000000001,
"vertices": [
"1:marko",
"2:lop",
"1:peter"
]
},
"2:lop": {
"weight": 0.4,
"vertices": [
"1:marko",
"2:lop"
]
},
"1:josh": {
"weight": 0.8,
"vertices": [
"1:marko",
"2:lop",
"1:josh"
]
},
"2:ripple": {
"weight": 1.8,
"vertices": [
"1:marko",
"2:lop",
"1:josh",
"2:ripple"
]
},
"1:vadas": {
"weight": 0.5,
"vertices": [
"1:marko",
"1:vadas"
]
}
},
"vertices": [
{
"id": "1:peter",
"label": "person",
"type": "vertex",
"properties": {
"name": "peter",
"age": 35,
"city": "Shanghai"
}
},
{
"id": "2:lop",
"label": "software",
"type": "vertex",
"properties": {
"name": "lop",
"lang": "java",
"price": 328
}
},
{
"id": "1:josh",
"label": "person",
"type": "vertex",
"properties": {
"name": "josh",
"age": 32,
"city": "Beijing"
}
},
{
"id": "2:ripple",
"label": "software",
"type": "vertex",
"properties": {
"name": "ripple",
"lang": "java",
"price": 199
}
},
{
"id": "1:marko",
"label": "person",
"type": "vertex",
"properties": {
"name": "marko",
"age": 29,
"city": "Beijing"
}
},
{
"id": "1:vadas",
"label": "person",
"type": "vertex",
"properties": {
"name": "vadas",
"age": 27,
"city": "Hongkong"
}
}
],
"edges": [
{
"id": "S1:josh>2>>S2:ripple",
"label": "created",
"type": "edge",
"outV": "1:josh",
"outVLabel": "person",
"inV": "2:ripple",
"inVLabel": "software",
"properties": {
"weight": 1.0,
"date": "20171210"
}
},
{
"id": "S1:marko>1>20160110>S1:vadas",
"label": "knows",
"type": "edge",
"outV": "1:marko",
"outVLabel": "person",
"inV": "1:vadas",
"inVLabel": "person",
"properties": {
"weight": 0.5,
"date": "20160110"
}
},
{
"id": "S1:josh>2>>S2:lop",
"label": "created",
"type": "edge",
"outV": "1:josh",
"outVLabel": "person",
"inV": "2:lop",
"inVLabel": "software",
"properties": {
"weight": 0.4,
"date": "20091111"
}
},
{
"id": "S1:marko>2>>S2:lop",
"label": "created",
"type": "edge",
"outV": "1:marko",
"outVLabel": "person",
"inV": "2:lop",
"inVLabel": "software",
"properties": {
"weight": 0.4,
"date": "20171210"
}
},
{
"id": "S1:peter>2>>S2:lop",
"label": "created",
"type": "edge",
"outV": "1:peter",
"outVLabel": "person",
"inV": "2:lop",
"inVLabel": "software",
"properties": {
"weight": 0.2,
"date": "20170324"
}
}
],
"measure": {
"edge_iters": 12,
"vertice_iters": 12,
"cost": 2
}
}
```
<img width="1480" alt="image" src="https://github.com/apache/incubator-hugegraph/assets/77946882/1cc33945-cf66-4e4b-99fe-5acae05c4710">
## Does this PR potentially affect the following parts?
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] Nope
- [ ] Dependencies (add/update license info) <!-- Don't forget to add/update the info in "LICENSE" & "NOTICE" files (both in root & dist module) -->
- [ ] Modify configurations
- [x] The public API
- [ ] Other affects (typed here)
## Documentation Status
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [x] `Doc - TODO` <!-- Your PR changes impact docs and you will update later -->
- [ ] `Doc - Done` <!-- Related docs have been already added or updated -->
- [ ] `Doc - No Need` <!-- Your PR changes don't impact/need docs -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] javeme commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "javeme (via GitHub)" <gi...@apache.org>.
javeme commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292183556
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KneighborAPI.java:
##########
@@ -111,15 +123,18 @@ public String post(@Context GraphManager manager,
E.checkArgument(request.step != null,
"The steps of request can't be null");
if (request.countOnly) {
- E.checkArgument(!request.withVertex && !request.withPath,
- "Can't return vertex or path when count only");
+ E.checkArgument(!request.withVertex && !request.withPath && !request.withEdge,
+ "Can't return vertex or path or edge when count only");
Review Comment:
`Can't return vertex, edge or path...`
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/FusiformSimilarityAPI.java:
##########
@@ -94,28 +95,38 @@ public String post(@Context GraphManager manager,
request.minNeighbors, request.alpha, request.minSimilars,
request.groupProperty, request.minGroups);
+ ApiMeasurer measure = new ApiMeasurer();
HugeGraph g = graph(manager, graph);
Iterator<Vertex> sources = request.sources.vertices(g);
E.checkArgument(sources != null && sources.hasNext(),
"The source vertices can't be empty");
FusiformSimilarityTraverser traverser =
- new FusiformSimilarityTraverser(g);
+ new FusiformSimilarityTraverser(g);
Review Comment:
seems one line is ok
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/SameNeighborsAPI.java:
##########
@@ -77,6 +89,81 @@ public String get(@Context GraphManager manager,
SameNeighborTraverser traverser = new SameNeighborTraverser(g);
Set<Id> neighbors = traverser.sameNeighbors(sourceId, targetId, dir,
edgeLabel, maxDegree, limit);
- return manager.serializer(g).writeList("same_neighbors", neighbors);
+
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ return manager.serializer(g, measure.measures())
+ .writeList("same_neighbors", neighbors);
+ }
+
+ @POST
+ @Timed
+ @Produces(APPLICATION_JSON_WITH_CHARSET)
+ public String sameNeighbors(@Context GraphManager manager,
+ @PathParam("graph") String graph,
+ Request request) {
+ LOG.debug("Graph [{}] get same neighbors among batch, '{}'", graph, request.toString());
+
+ ApiMeasurer measure = new ApiMeasurer();
+
+ Directions dir = Directions.convert(EdgeAPI.parseDirection(request.direction));
+ HugeGraph g = graph(manager, graph);
+ SameNeighborTraverser traverser = new SameNeighborTraverser(g);
+
+ List<Object> vertexList = request.vertexList;
+ E.checkArgument(vertexList.size() >= 2, "vertex_list size can't " +
+ "be less than 2");
+
+ List<Id> vertexIds = new ArrayList<>();
+ for (Object obj : vertexList) {
+ vertexIds.add(HugeVertex.getIdValue(obj));
+ }
+
+ Set<Id> neighbors = traverser.sameNeighbors(vertexIds, dir, request.labels,
+ request.maxDegree, request.limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ Iterator<?> iterVertex;
+ Set<Id> ids = new HashSet<>(neighbors);
+ ids.addAll(vertexIds);
+ if (request.withVertex && !ids.isEmpty()) {
+ iterVertex = g.vertices(ids.toArray());
+ } else {
+ iterVertex = ids.iterator();
+ }
+ return manager.serializer(g, measure.measures())
+ .writeMap(ImmutableMap.of("same_neighbors", neighbors,
+ "vertices", iterVertex));
+ }
+
+ private static class Request {
+ @JsonProperty("max_degree")
+ public long maxDegree = Long.parseLong(DEFAULT_MAX_DEGREE);
+ @JsonProperty("limit")
+ public int limit = Integer.parseInt(DEFAULT_ELEMENTS_LIMIT);
+ @JsonProperty("vertex_list")
+ private List<Object> vertexList;
+ @JsonProperty("direction")
+ private String direction;
+ @JsonProperty("labels")
+ private List<String> labels;
+ @JsonProperty("with_vertex")
+ private boolean withVertex = false;
+
+ @Override
+ public String toString() {
Review Comment:
the way of this method implement is expected to keep the same style as other Request class
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/records/SingleWayMultiPathsRecords.java:
##########
@@ -44,7 +45,7 @@ public abstract class SingleWayMultiPathsRecords extends AbstractRecords {
private final int sourceCode;
private final boolean nearest;
private final IntSet accessedVertices;
-
+ private final HugeTraverser.EdgeRecord edgeRecord;
Review Comment:
prefer to rename to edgeResults
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizePathsTraverser.java:
##########
@@ -22,25 +22,54 @@
import java.util.List;
import java.util.Map;
-import jakarta.ws.rs.core.MultivaluedMap;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.backend.id.Id;
-import org.apache.tinkerpop.gremlin.structure.Edge;
-import org.apache.tinkerpop.gremlin.structure.Vertex;
-
import org.apache.hugegraph.structure.HugeEdge;
import org.apache.hugegraph.structure.HugeVertex;
import org.apache.hugegraph.traversal.algorithm.steps.WeightedEdgeStep;
import org.apache.hugegraph.util.CollectionUtil;
import org.apache.hugegraph.util.E;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
+import jakarta.ws.rs.core.MultivaluedMap;
+
public class CustomizePathsTraverser extends HugeTraverser {
+ private final EdgeRecord edgeRecord;
Review Comment:
expect a blank line, and prefer to rename to edgeResults.
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizePathsTraverser.java:
##########
@@ -133,36 +171,13 @@ public List<Path> customizedPaths(Iterator<Vertex> vertices,
return paths;
}
- public static List<Path> topNPath(List<Path> paths,
- boolean incr, long limit) {
- paths.sort((p1, p2) -> {
- WeightPath wp1 = (WeightPath) p1;
- WeightPath wp2 = (WeightPath) p2;
- int result = Double.compare(wp1.totalWeight(), wp2.totalWeight());
- return incr ? result : -result;
- });
-
- if (limit == NO_LIMIT || paths.size() <= limit) {
- return paths;
- }
- return paths.subList(0, (int) limit);
- }
-
- private static List<Node> sample(List<Node> nodes, long sample) {
- if (nodes.size() <= sample) {
- return nodes;
- }
- List<Node> result = newList((int) sample);
- int size = nodes.size();
- for (int random : CollectionUtil.randomSet(0, size, (int) sample)) {
- result.add(nodes.get(random));
- }
- return result;
+ public EdgeRecord getEdgeRecord() {
Review Comment:
ditto
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -224,8 +220,7 @@ protected void addCount(String key, long value) {
if (current == null) {
measures.put(key, new MutableLong(value));
} else if (current instanceof MutableLong) {
- MutableLong currentMutableLong = (MutableLong) current;
- currentMutableLong.add(value);
+ ((MutableLong) measures.computeIfAbsent(key, MutableLong::new)).add(value);
} else if (current instanceof Long) {
Review Comment:
add else branch then throw an exception
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizedCrosspointsTraverser.java:
##########
@@ -24,26 +24,82 @@
import java.util.Set;
import java.util.stream.Collectors;
-import jakarta.ws.rs.core.MultivaluedMap;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.backend.id.Id;
+import org.apache.hugegraph.structure.HugeEdge;
+import org.apache.hugegraph.structure.HugeVertex;
import org.apache.hugegraph.traversal.algorithm.steps.EdgeStep;
import org.apache.hugegraph.type.define.Directions;
+import org.apache.hugegraph.util.CollectionUtil;
+import org.apache.hugegraph.util.E;
import org.apache.tinkerpop.gremlin.structure.Edge;
import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.structure.HugeEdge;
-import org.apache.hugegraph.structure.HugeVertex;
-import org.apache.hugegraph.util.CollectionUtil;
-import org.apache.hugegraph.util.E;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
+import jakarta.ws.rs.core.MultivaluedMap;
+
public class CustomizedCrosspointsTraverser extends HugeTraverser {
+ private final EdgeRecord edgeRecord;
Review Comment:
ditto
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/TemplatePathsAPI.java:
##########
@@ -74,50 +90,50 @@ public String post(@Context GraphManager manager,
LOG.debug("Graph [{}] get template paths from source vertices '{}', " +
"target vertices '{}', with steps '{}', " +
- "capacity '{}', limit '{}' and with_vertex '{}'",
+ "capacity '{}', limit '{}', with vertex '{}' and with edge '{}'",
Review Comment:
ditto
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/CustomizedPathsAPI.java:
##########
@@ -30,40 +30,50 @@
import java.util.Map;
import java.util.Set;
-import io.swagger.v3.oas.annotations.tags.Tag;
-import jakarta.inject.Singleton;
-import jakarta.ws.rs.Consumes;
-import jakarta.ws.rs.POST;
-import jakarta.ws.rs.Path;
-import jakarta.ws.rs.PathParam;
-import jakarta.ws.rs.Produces;
-import jakarta.ws.rs.core.Context;
-
-import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.core.GraphManager;
-import org.slf4j.Logger;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.api.API;
import org.apache.hugegraph.backend.id.Id;
-import org.apache.hugegraph.backend.query.QueryResults;
+import org.apache.hugegraph.core.GraphManager;
import org.apache.hugegraph.traversal.algorithm.CustomizePathsTraverser;
import org.apache.hugegraph.traversal.algorithm.HugeTraverser;
import org.apache.hugegraph.traversal.algorithm.steps.WeightedEdgeStep;
import org.apache.hugegraph.type.define.Directions;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.slf4j.Logger;
+
import com.codahale.metrics.annotation.Timed;
import com.fasterxml.jackson.annotation.JsonAlias;
import com.fasterxml.jackson.annotation.JsonProperty;
+import io.swagger.v3.oas.annotations.tags.Tag;
+import jakarta.inject.Singleton;
+import jakarta.ws.rs.Consumes;
+import jakarta.ws.rs.POST;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.PathParam;
+import jakarta.ws.rs.Produces;
+import jakarta.ws.rs.core.Context;
+
@Path("graphs/{graph}/traversers/customizedpaths")
@Singleton
@Tag(name = "CustomizedPathsAPI")
public class CustomizedPathsAPI extends API {
private static final Logger LOG = Log.logger(CustomizedPathsAPI.class);
+ private static List<WeightedEdgeStep> step(HugeGraph graph,
+ PathRequest req) {
Review Comment:
also keep request name style?
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/PathsAPI.java:
##########
@@ -120,35 +125,51 @@ public String post(@Context GraphManager manager,
LOG.debug("Graph [{}] get paths from source vertices '{}', target " +
"vertices '{}', with step '{}', max depth '{}', " +
- "capacity '{}', limit '{}' and with_vertex '{}'",
+ "capacity '{}', limit '{}', with vertex '{}' and with edge '{}'",
Review Comment:
keep the unified style `with_vertex`, the same to other places
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KoutAPI.java:
##########
@@ -116,32 +129,35 @@ public String post(@Context GraphManager manager,
E.checkArgument(request.step != null,
"The steps of request can't be null");
if (request.countOnly) {
- E.checkArgument(!request.withVertex && !request.withPath,
- "Can't return vertex or path when count only");
+ E.checkArgument(!request.withVertex && !request.withPath && !request.withEdge,
+ "Can't return vertex or path or edge when count only");
Review Comment:
ditto, also check other places
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -44,29 +44,25 @@
public class API {
- protected static final Logger LOG = Log.logger(API.class);
-
public static final String CHARSET = "UTF-8";
-
public static final String TEXT_PLAIN = MediaType.TEXT_PLAIN;
public static final String APPLICATION_JSON = MediaType.APPLICATION_JSON;
public static final String APPLICATION_JSON_WITH_CHARSET =
- APPLICATION_JSON + ";charset=" + CHARSET;
+ APPLICATION_JSON + ";charset=" + CHARSET;
public static final String JSON = MediaType.APPLICATION_JSON_TYPE
- .getSubtype();
-
+ .getSubtype();
Review Comment:
can we keep the origin style
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/PathTraverser.java:
##########
@@ -52,10 +50,11 @@ public abstract class PathTraverser {
protected Set<HugeTraverser.Path> paths;
protected TraverseStrategy traverseStrategy;
+ protected HugeTraverser.EdgeRecord edgeRecord;
Review Comment:
just import EdgeRecord to keep the same style?
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/SameNeighborsAPI.java:
##########
@@ -77,6 +89,81 @@ public String get(@Context GraphManager manager,
SameNeighborTraverser traverser = new SameNeighborTraverser(g);
Set<Id> neighbors = traverser.sameNeighbors(sourceId, targetId, dir,
edgeLabel, maxDegree, limit);
- return manager.serializer(g).writeList("same_neighbors", neighbors);
+
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ return manager.serializer(g, measure.measures())
+ .writeList("same_neighbors", neighbors);
+ }
+
+ @POST
+ @Timed
+ @Produces(APPLICATION_JSON_WITH_CHARSET)
+ public String sameNeighbors(@Context GraphManager manager,
+ @PathParam("graph") String graph,
+ Request request) {
+ LOG.debug("Graph [{}] get same neighbors among batch, '{}'", graph, request.toString());
+
+ ApiMeasurer measure = new ApiMeasurer();
+
+ Directions dir = Directions.convert(EdgeAPI.parseDirection(request.direction));
+ HugeGraph g = graph(manager, graph);
+ SameNeighborTraverser traverser = new SameNeighborTraverser(g);
+
+ List<Object> vertexList = request.vertexList;
+ E.checkArgument(vertexList.size() >= 2, "vertex_list size can't " +
+ "be less than 2");
+
+ List<Id> vertexIds = new ArrayList<>();
+ for (Object obj : vertexList) {
+ vertexIds.add(HugeVertex.getIdValue(obj));
+ }
+
+ Set<Id> neighbors = traverser.sameNeighbors(vertexIds, dir, request.labels,
+ request.maxDegree, request.limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ Iterator<?> iterVertex;
+ Set<Id> ids = new HashSet<>(neighbors);
+ ids.addAll(vertexIds);
+ if (request.withVertex && !ids.isEmpty()) {
+ iterVertex = g.vertices(ids.toArray());
+ } else {
+ iterVertex = ids.iterator();
+ }
+ return manager.serializer(g, measure.measures())
+ .writeMap(ImmutableMap.of("same_neighbors", neighbors,
+ "vertices", iterVertex));
+ }
+
+ private static class Request {
Review Comment:
expect a blank line
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/ShortestPathTraverser.java:
##########
@@ -126,27 +135,36 @@ public PathSet allShortestPaths(Id sourceV, Id targetV, Directions dir,
checkCapacity(traverser.capacity, traverser.accessed(),
"shortest path");
}
+
+ this.vertexIterCounter.addAndGet(traverser.vertexCount);
+ this.edgeIterCounter.addAndGet(traverser.record.accessed());
+
+ paths.setEdges(traverser.edgeRecord.getEdges(paths));
return paths;
}
private class Traverser {
private final ShortestPathRecords record;
+ private final EdgeRecord edgeRecord;
Review Comment:
rename to pathResults and edgeResults
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/records/SingleWayMultiPathsRecords.java:
##########
@@ -176,6 +178,10 @@ protected final Stack<Record> records() {
return this.records;
}
+ public HugeTraverser.EdgeRecord getEdgeIdRecord() {
Review Comment:
rename to edgeResults()?
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KoutAPI.java:
##########
@@ -99,8 +105,15 @@ public String get(@Context GraphManager manager,
try (KoutTraverser traverser = new KoutTraverser(g)) {
ids = traverser.kout(sourceId, dir, edgeLabel, depth,
nearest, maxDegree, capacity, limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+ }
+
+ if (count_only) {
+ return manager.serializer(g, measure.measures()).writeMap(ImmutableMap.of(
Review Comment:
prefer to wrap line before writeMap()
##########
hugegraph-api/src/main/java/org/apache/hugegraph/config/ServerOptions.java:
##########
@@ -256,12 +218,24 @@ public static synchronized ServerOptions instance() {
null,
"hugegraph"
);
-
public static final ConfigOption<Boolean> ENABLE_DYNAMIC_CREATE_DROP =
new ConfigOption<>(
"graphs.enable_dynamic_create_drop",
"Whether to enable create or drop graph dynamically.",
disallowEmpty(),
true
);
+ private static volatile ServerOptions instance;
+
+ private ServerOptions() {
+ super();
+ }
+
+ public static synchronized ServerOptions instance() {
+ if (instance == null) {
+ instance = new ServerOptions();
+ instance.registerOptions();
+ }
+ return instance;
+ }
Review Comment:
prefer to don't update style of this file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291011547
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -186,8 +185,56 @@ public static boolean checkAndParseAction(String action) {
} else if (action.equals(ACTION_ELIMINATE)) {
return false;
} else {
- throw new NotSupportedException(
- String.format("Not support action '%s'", action));
+ throw new NotSupportedException(String.format("Not support action '%s'", action));
+ }
+ }
+
+ public static class ApiMeasurer {
+
+ public static final String EDGE_ITER = "edge_iterations";
+ public static final String VERTICE_ITER = "vertice_iterations";
+ public static final String COST = "cost";
+ private final long timeStart;
+ private final Map<String, Object> measures;
+
+ public ApiMeasurer() {
+ this.timeStart = System.currentTimeMillis();
+ this.measures = InsertionOrderUtil.newMap();
+ }
+
+ public Map<String, Object> measures() {
+ measures.put(COST, System.currentTimeMillis() - timeStart);
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292234833
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizedCrosspointsTraverser.java:
##########
@@ -24,26 +24,82 @@
import java.util.Set;
import java.util.stream.Collectors;
-import jakarta.ws.rs.core.MultivaluedMap;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.backend.id.Id;
+import org.apache.hugegraph.structure.HugeEdge;
+import org.apache.hugegraph.structure.HugeVertex;
import org.apache.hugegraph.traversal.algorithm.steps.EdgeStep;
import org.apache.hugegraph.type.define.Directions;
+import org.apache.hugegraph.util.CollectionUtil;
+import org.apache.hugegraph.util.E;
import org.apache.tinkerpop.gremlin.structure.Edge;
import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.structure.HugeEdge;
-import org.apache.hugegraph.structure.HugeVertex;
-import org.apache.hugegraph.util.CollectionUtil;
-import org.apache.hugegraph.util.E;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
+import jakarta.ws.rs.core.MultivaluedMap;
+
public class CustomizedCrosspointsTraverser extends HugeTraverser {
+ private final EdgeRecord edgeRecord;
Review Comment:
fixed
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizePathsTraverser.java:
##########
@@ -22,25 +22,54 @@
import java.util.List;
import java.util.Map;
-import jakarta.ws.rs.core.MultivaluedMap;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.backend.id.Id;
-import org.apache.tinkerpop.gremlin.structure.Edge;
-import org.apache.tinkerpop.gremlin.structure.Vertex;
-
import org.apache.hugegraph.structure.HugeEdge;
import org.apache.hugegraph.structure.HugeVertex;
import org.apache.hugegraph.traversal.algorithm.steps.WeightedEdgeStep;
import org.apache.hugegraph.util.CollectionUtil;
import org.apache.hugegraph.util.E;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
+import jakarta.ws.rs.core.MultivaluedMap;
+
public class CustomizePathsTraverser extends HugeTraverser {
+ private final EdgeRecord edgeRecord;
Review Comment:
fixed
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/PathTraverser.java:
##########
@@ -52,10 +50,11 @@ public abstract class PathTraverser {
protected Set<HugeTraverser.Path> paths;
protected TraverseStrategy traverseStrategy;
+ protected HugeTraverser.EdgeRecord edgeRecord;
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] github-code-scanning[bot] commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "github-code-scanning[bot] (via GitHub)" <gi...@apache.org>.
github-code-scanning[bot] commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1277200190
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/SameNeighborTraverser.java:
##########
@@ -46,15 +48,60 @@
Id labelId = this.getEdgeLabelId(label);
Set<Id> sourceNeighbors = IteratorUtils.set(this.adjacentVertices(
- vertex, direction, labelId, degree));
+ vertex, direction, labelId, degree));
Set<Id> targetNeighbors = IteratorUtils.set(this.adjacentVertices(
- other, direction, labelId, degree));
+ other, direction, labelId, degree));
Set<Id> sameNeighbors = (Set<Id>) CollectionUtil.intersect(
- sourceNeighbors, targetNeighbors);
+ sourceNeighbors, targetNeighbors);
+
+ this.vertexIterCounter.addAndGet(2L);
+ this.edgeIterCounter.addAndGet(sourceNeighbors.size());
+ this.edgeIterCounter.addAndGet(targetNeighbors.size());
+
if (limit != NO_LIMIT) {
int end = Math.min(sameNeighbors.size(), limit);
sameNeighbors = CollectionUtil.subSet(sameNeighbors, 0, end);
}
return sameNeighbors;
}
+
+ public Set<Id> sameNeighbors(List<Id> vertexIds, Directions direction,
+ List<String> labels, long degree, long limit) {
+ E.checkNotNull(vertexIds, "vertex ids");
+ E.checkArgument(vertexIds.size() >= 2, "vertex_list size can't " +
+ "be less than 2");
+ for (Id id : vertexIds) {
+ this.checkVertexExist(id, "vertex");
+ }
+ E.checkNotNull(direction, "direction");
+ checkDegree(degree);
+ checkLimit(limit);
+
+ List<Id> labelsId = new ArrayList<>();
+ if (labels != null) {
+ for (String label : labels) {
+ labelsId.add(this.getEdgeLabelId(label));
+ }
+ }
+
+ Set<Id> sameNeighbors = new HashSet<>();
+ for (int i = 0; i < vertexIds.size(); i++) {
+ Set<Id> vertexNeighbors = IteratorUtils.set(this.adjacentVertices(
+ vertexIds.get(i), direction, labelsId, degree));
+ if (i == 0) {
+ sameNeighbors = vertexNeighbors;
+ } else {
+ sameNeighbors = (Set<Id>) CollectionUtil.intersect(
+ sameNeighbors, vertexNeighbors);
+ }
+ this.vertexIterCounter.addAndGet(1L);
+ this.edgeIterCounter.addAndGet(vertexNeighbors.size());
+ }
+
+ if (limit != NO_LIMIT) {
+ int end = Math.min(sameNeighbors.size(), (int) limit);
Review Comment:
## User-controlled data in numeric cast
This cast to a narrower type depends on a [user-provided value](1), potentially causing truncation.
[Show more details](https://github.com/apache/incubator-hugegraph/security/code-scanning/45)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] zyxxoo commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "zyxxoo (via GitHub)" <gi...@apache.org>.
zyxxoo commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1290881610
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -186,8 +185,56 @@ public static boolean checkAndParseAction(String action) {
} else if (action.equals(ACTION_ELIMINATE)) {
return false;
} else {
- throw new NotSupportedException(
- String.format("Not support action '%s'", action));
+ throw new NotSupportedException(String.format("Not support action '%s'", action));
+ }
+ }
+
+ public static class ApiMeasurer {
+
+ public static final String EDGE_ITER = "edge_iterations";
+ public static final String VERTICE_ITER = "vertice_iterations";
+ public static final String COST = "cost";
+ private final long timeStart;
+ private final Map<String, Object> measures;
+
+ public ApiMeasurer() {
+ this.timeStart = System.currentTimeMillis();
+ this.measures = InsertionOrderUtil.newMap();
+ }
+
+ public Map<String, Object> measures() {
+ measures.put(COST, System.currentTimeMillis() - timeStart);
Review Comment:
compute duration use System.nanoTime perfered
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -186,8 +185,56 @@ public static boolean checkAndParseAction(String action) {
} else if (action.equals(ACTION_ELIMINATE)) {
return false;
} else {
- throw new NotSupportedException(
- String.format("Not support action '%s'", action));
+ throw new NotSupportedException(String.format("Not support action '%s'", action));
+ }
+ }
+
+ public static class ApiMeasurer {
+
+ public static final String EDGE_ITER = "edge_iterations";
+ public static final String VERTICE_ITER = "vertice_iterations";
+ public static final String COST = "cost";
+ private final long timeStart;
+ private final Map<String, Object> measures;
+
+ public ApiMeasurer() {
+ this.timeStart = System.currentTimeMillis();
+ this.measures = InsertionOrderUtil.newMap();
+ }
+
+ public Map<String, Object> measures() {
+ measures.put(COST, System.currentTimeMillis() - timeStart);
+ return measures;
+ }
+
+ public void put(String key, String value) {
+ this.measures.put(key, value);
+ }
+
+ public void put(String key, long value) {
+ this.measures.put(key, value);
+ }
+
+ public void put(String key, int value) {
+ this.measures.put(key, value);
+ }
+
+ protected void addCount(String key, long value) {
+ Object current = measures.get(key);
+ if (current == null) {
+ measures.put(key, new MutableLong(value));
+ } else if (current instanceof MutableLong) {
+ MutableLong currentMutableLong = (MutableLong) current;
+ currentMutableLong.add(value);
+ } else if (current instanceof Long) {
Review Comment:
measures.computeIfAbsent(key, MutableLong.new).add(value)
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/HugeTraverser.java:
##########
@@ -82,13 +82,16 @@ public class HugeTraverser {
public static final String DEFAULT_SAMPLE = "100";
public static final String DEFAULT_WEIGHT = "0";
public static final int DEFAULT_MAX_DEPTH = 5000;
-
- protected static final int MAX_VERTICES = 10;
-
// Empirical value of scan limit, with which results can be returned in 3s
public static final String DEFAULT_PAGE_LIMIT = "100000";
-
public static final long NO_LIMIT = -1L;
+ protected static final Logger LOG = Log.logger(HugeTraverser.class);
+ protected static final int MAX_VERTICES = 10;
+ private static CollectionFactory collectionFactory;
+ private final HugeGraph graph;
+ // for apimeasure
+ public AtomicLong edgeIterCounter = new AtomicLong(0);
Review Comment:
Wouldn't it be better for this class to have no side effects?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1278763706
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/CrosspointsAPI.java:
##########
@@ -84,6 +85,9 @@ public String get(@Context GraphManager manager,
dir, edgeLabel, depth,
maxDegree, capacity,
limit);
- return manager.serializer(g).writePaths("crosspoints", paths, true);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+ return manager.serializer(g, measure.getResult())
Review Comment:
Rejected:
1. `ApiMeasurer` should be passed in from an outside api implementation so that it can carry data
2. not all apis need `ApiMeasurer` and keeping the `ApiMeasurer` null for those APIs can prevent `ApiMeasurer` from being serialized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] zyxxoo commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "zyxxoo (via GitHub)" <gi...@apache.org>.
zyxxoo commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1290951756
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizedCrosspointsTraverser.java:
##########
@@ -24,26 +24,82 @@
import java.util.Set;
import java.util.stream.Collectors;
-import jakarta.ws.rs.core.MultivaluedMap;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.backend.id.Id;
+import org.apache.hugegraph.structure.HugeEdge;
+import org.apache.hugegraph.structure.HugeVertex;
import org.apache.hugegraph.traversal.algorithm.steps.EdgeStep;
import org.apache.hugegraph.type.define.Directions;
+import org.apache.hugegraph.util.CollectionUtil;
+import org.apache.hugegraph.util.E;
import org.apache.tinkerpop.gremlin.structure.Edge;
import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.structure.HugeEdge;
-import org.apache.hugegraph.structure.HugeVertex;
-import org.apache.hugegraph.util.CollectionUtil;
-import org.apache.hugegraph.util.E;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
+import jakarta.ws.rs.core.MultivaluedMap;
+
public class CustomizedCrosspointsTraverser extends HugeTraverser {
+ private final EdgeRecord edgeRecord;
+
public CustomizedCrosspointsTraverser(HugeGraph graph) {
super(graph);
+ this.edgeRecord = new EdgeRecord(false);
+ }
+
+ private static CrosspointsPaths intersectionPaths(List<HugeVertex> sources,
+ List<Path> paths,
+ long limit) {
+ // Split paths by end vertices
+ MultivaluedMap<Id, Id> endVertices = newMultivalueMap();
+ for (Path path : paths) {
+ List<Id> vertices = path.vertices();
+ int length = vertices.size();
+ endVertices.add(vertices.get(0), vertices.get(length - 1));
+ }
+
+ Set<Id> sourceIds = sources.stream().map(HugeVertex::id)
+ .collect(Collectors.toSet());
+ Set<Id> ids = endVertices.keySet();
+ if (sourceIds.size() != ids.size() || !sourceIds.containsAll(ids)) {
+ return CrosspointsPaths.EMPTY;
+ }
+
+ // Get intersection of end vertices
+ Collection<Id> intersection = null;
+ for (List<Id> ends : endVertices.values()) {
+ if (intersection == null) {
+ intersection = ends;
+ } else {
+ intersection = CollectionUtil.intersect(intersection, ends);
+ }
+ if (intersection == null || intersection.isEmpty()) {
+ return CrosspointsPaths.EMPTY;
+ }
+ }
+ assert intersection != null;
+ // Limit intersection number to limit crosspoints vertices in result
+ int size = intersection.size();
+ if (limit != NO_LIMIT && size > limit) {
+ intersection = newList(intersection).subList(0, size - 1);
+ }
+
+ // Filter intersection paths
+ List<Path> results = newList();
+ for (Path path : paths) {
+ List<Id> vertices = path.vertices();
+ int length = vertices.size();
+ if (intersection.contains(vertices.get(length - 1))) {
Review Comment:
try keep origin code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1278759589
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KneighborAPI.java:
##########
@@ -94,8 +100,14 @@ public String get(@Context GraphManager manager,
try (KneighborTraverser traverser = new KneighborTraverser(g)) {
ids = traverser.kneighbor(source, dir, edgeLabel,
depth, maxDegree, limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+ }
+ if (countOnly) {
+ return manager.serializer(g, measure.getResult()).writeMap(
Review Comment:
fixed
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292234800
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/SameNeighborsAPI.java:
##########
@@ -77,6 +89,81 @@ public String get(@Context GraphManager manager,
SameNeighborTraverser traverser = new SameNeighborTraverser(g);
Set<Id> neighbors = traverser.sameNeighbors(sourceId, targetId, dir,
edgeLabel, maxDegree, limit);
- return manager.serializer(g).writeList("same_neighbors", neighbors);
+
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ return manager.serializer(g, measure.measures())
+ .writeList("same_neighbors", neighbors);
+ }
+
+ @POST
+ @Timed
+ @Produces(APPLICATION_JSON_WITH_CHARSET)
+ public String sameNeighbors(@Context GraphManager manager,
+ @PathParam("graph") String graph,
+ Request request) {
+ LOG.debug("Graph [{}] get same neighbors among batch, '{}'", graph, request.toString());
+
+ ApiMeasurer measure = new ApiMeasurer();
+
+ Directions dir = Directions.convert(EdgeAPI.parseDirection(request.direction));
+ HugeGraph g = graph(manager, graph);
+ SameNeighborTraverser traverser = new SameNeighborTraverser(g);
+
+ List<Object> vertexList = request.vertexList;
+ E.checkArgument(vertexList.size() >= 2, "vertex_list size can't " +
+ "be less than 2");
+
+ List<Id> vertexIds = new ArrayList<>();
+ for (Object obj : vertexList) {
+ vertexIds.add(HugeVertex.getIdValue(obj));
+ }
+
+ Set<Id> neighbors = traverser.sameNeighbors(vertexIds, dir, request.labels,
+ request.maxDegree, request.limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ Iterator<?> iterVertex;
+ Set<Id> ids = new HashSet<>(neighbors);
+ ids.addAll(vertexIds);
+ if (request.withVertex && !ids.isEmpty()) {
+ iterVertex = g.vertices(ids.toArray());
+ } else {
+ iterVertex = ids.iterator();
+ }
+ return manager.serializer(g, measure.measures())
+ .writeMap(ImmutableMap.of("same_neighbors", neighbors,
+ "vertices", iterVertex));
+ }
+
+ private static class Request {
+ @JsonProperty("max_degree")
+ public long maxDegree = Long.parseLong(DEFAULT_MAX_DEGREE);
+ @JsonProperty("limit")
+ public int limit = Integer.parseInt(DEFAULT_ELEMENTS_LIMIT);
+ @JsonProperty("vertex_list")
+ private List<Object> vertexList;
+ @JsonProperty("direction")
+ private String direction;
+ @JsonProperty("labels")
+ private List<String> labels;
+ @JsonProperty("with_vertex")
+ private boolean withVertex = false;
+
+ @Override
+ public String toString() {
Review Comment:
fixed
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/TemplatePathsAPI.java:
##########
@@ -74,50 +90,50 @@ public String post(@Context GraphManager manager,
LOG.debug("Graph [{}] get template paths from source vertices '{}', " +
"target vertices '{}', with steps '{}', " +
- "capacity '{}', limit '{}' and with_vertex '{}'",
+ "capacity '{}', limit '{}', with vertex '{}' and with edge '{}'",
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292234712
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KneighborAPI.java:
##########
@@ -111,15 +123,18 @@ public String post(@Context GraphManager manager,
E.checkArgument(request.step != null,
"The steps of request can't be null");
if (request.countOnly) {
- E.checkArgument(!request.withVertex && !request.withPath,
- "Can't return vertex or path when count only");
+ E.checkArgument(!request.withVertex && !request.withPath && !request.withEdge,
+ "Can't return vertex or path or edge when count only");
Review Comment:
fixed
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/PathsAPI.java:
##########
@@ -120,35 +125,51 @@ public String post(@Context GraphManager manager,
LOG.debug("Graph [{}] get paths from source vertices '{}', target " +
"vertices '{}', with step '{}', max depth '{}', " +
- "capacity '{}', limit '{}' and with_vertex '{}'",
+ "capacity '{}', limit '{}', with vertex '{}' and with edge '{}'",
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292234628
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -44,29 +44,25 @@
public class API {
- protected static final Logger LOG = Log.logger(API.class);
-
public static final String CHARSET = "UTF-8";
-
public static final String TEXT_PLAIN = MediaType.TEXT_PLAIN;
public static final String APPLICATION_JSON = MediaType.APPLICATION_JSON;
public static final String APPLICATION_JSON_WITH_CHARSET =
- APPLICATION_JSON + ";charset=" + CHARSET;
+ APPLICATION_JSON + ";charset=" + CHARSET;
public static final String JSON = MediaType.APPLICATION_JSON_TYPE
- .getSubtype();
-
+ .getSubtype();
Review Comment:
fixed
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -224,8 +220,7 @@ protected void addCount(String key, long value) {
if (current == null) {
measures.put(key, new MutableLong(value));
} else if (current instanceof MutableLong) {
- MutableLong currentMutableLong = (MutableLong) current;
- currentMutableLong.add(value);
+ ((MutableLong) measures.computeIfAbsent(key, MutableLong::new)).add(value);
} else if (current instanceof Long) {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1278763706
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/CrosspointsAPI.java:
##########
@@ -84,6 +85,9 @@ public String get(@Context GraphManager manager,
dir, edgeLabel, depth,
maxDegree, capacity,
limit);
- return manager.serializer(g).writePaths("crosspoints", paths, true);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+ return manager.serializer(g, measure.getResult())
Review Comment:
Rejected:
1. `ApiMeasurer` should be passed in from an outside api implementation so that it can carry data
2. not all apis need ApiMeasurer and keeping the ApiMeasurer null for those APIs can prevent `ApiMeasurer` from being serialized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] imbajin commented on pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#issuecomment-1682704475
Good job, thanks for ur contributuion & reaction
'll check this pr carefully later, @zyxxoo could ensure it now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1297902672
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizedCrosspointsTraverser.java:
##########
@@ -24,26 +24,82 @@
import java.util.Set;
import java.util.stream.Collectors;
-import jakarta.ws.rs.core.MultivaluedMap;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.backend.id.Id;
+import org.apache.hugegraph.structure.HugeEdge;
+import org.apache.hugegraph.structure.HugeVertex;
import org.apache.hugegraph.traversal.algorithm.steps.EdgeStep;
import org.apache.hugegraph.type.define.Directions;
+import org.apache.hugegraph.util.CollectionUtil;
+import org.apache.hugegraph.util.E;
import org.apache.tinkerpop.gremlin.structure.Edge;
import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.structure.HugeEdge;
-import org.apache.hugegraph.structure.HugeVertex;
-import org.apache.hugegraph.util.CollectionUtil;
-import org.apache.hugegraph.util.E;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
+import jakarta.ws.rs.core.MultivaluedMap;
+
public class CustomizedCrosspointsTraverser extends HugeTraverser {
+ private final EdgeRecord edgeRecord;
+
public CustomizedCrosspointsTraverser(HugeGraph graph) {
super(graph);
+ this.edgeRecord = new EdgeRecord(false);
+ }
+
+ private static CrosspointsPaths intersectionPaths(List<HugeVertex> sources,
+ List<Path> paths,
+ long limit) {
+ // Split paths by end vertices
+ MultivaluedMap<Id, Id> endVertices = newMultivalueMap();
+ for (Path path : paths) {
+ List<Id> vertices = path.vertices();
+ int length = vertices.size();
+ endVertices.add(vertices.get(0), vertices.get(length - 1));
+ }
+
+ Set<Id> sourceIds = sources.stream().map(HugeVertex::id)
+ .collect(Collectors.toSet());
+ Set<Id> ids = endVertices.keySet();
+ if (sourceIds.size() != ids.size() || !sourceIds.containsAll(ids)) {
+ return CrosspointsPaths.EMPTY;
+ }
+
+ // Get intersection of end vertices
+ Collection<Id> intersection = null;
+ for (List<Id> ends : endVertices.values()) {
+ if (intersection == null) {
+ intersection = ends;
+ } else {
+ intersection = CollectionUtil.intersect(intersection, ends);
+ }
+ if (intersection == null || intersection.isEmpty()) {
+ return CrosspointsPaths.EMPTY;
+ }
+ }
+ assert intersection != null;
+ // Limit intersection number to limit crosspoints vertices in result
+ int size = intersection.size();
+ if (limit != NO_LIMIT && size > limit) {
+ intersection = newList(intersection).subList(0, size - 1);
+ }
+
+ // Filter intersection paths
+ List<Path> results = newList();
+ for (Path path : paths) {
+ List<Id> vertices = path.vertices();
+ int length = vertices.size();
+ if (intersection.contains(vertices.get(length - 1))) {
Review Comment:
Do you mean to leave this static method in its original place?
I just formatted it automatically. The content of this method is the same as the original, and only the location has changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] imbajin commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1299112364
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizedCrosspointsTraverser.java:
##########
@@ -24,26 +24,82 @@
import java.util.Set;
import java.util.stream.Collectors;
-import jakarta.ws.rs.core.MultivaluedMap;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.backend.id.Id;
+import org.apache.hugegraph.structure.HugeEdge;
+import org.apache.hugegraph.structure.HugeVertex;
import org.apache.hugegraph.traversal.algorithm.steps.EdgeStep;
import org.apache.hugegraph.type.define.Directions;
+import org.apache.hugegraph.util.CollectionUtil;
+import org.apache.hugegraph.util.E;
import org.apache.tinkerpop.gremlin.structure.Edge;
import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.structure.HugeEdge;
-import org.apache.hugegraph.structure.HugeVertex;
-import org.apache.hugegraph.util.CollectionUtil;
-import org.apache.hugegraph.util.E;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
+import jakarta.ws.rs.core.MultivaluedMap;
+
public class CustomizedCrosspointsTraverser extends HugeTraverser {
+ private final EdgeRecord edgeRecord;
+
public CustomizedCrosspointsTraverser(HugeGraph graph) {
super(graph);
+ this.edgeRecord = new EdgeRecord(false);
+ }
+
+ private static CrosspointsPaths intersectionPaths(List<HugeVertex> sources,
+ List<Path> paths,
+ long limit) {
+ // Split paths by end vertices
+ MultivaluedMap<Id, Id> endVertices = newMultivalueMap();
+ for (Path path : paths) {
+ List<Id> vertices = path.vertices();
+ int length = vertices.size();
+ endVertices.add(vertices.get(0), vertices.get(length - 1));
+ }
+
+ Set<Id> sourceIds = sources.stream().map(HugeVertex::id)
+ .collect(Collectors.toSet());
+ Set<Id> ids = endVertices.keySet();
+ if (sourceIds.size() != ids.size() || !sourceIds.containsAll(ids)) {
+ return CrosspointsPaths.EMPTY;
+ }
+
+ // Get intersection of end vertices
+ Collection<Id> intersection = null;
+ for (List<Id> ends : endVertices.values()) {
+ if (intersection == null) {
+ intersection = ends;
+ } else {
+ intersection = CollectionUtil.intersect(intersection, ends);
+ }
+ if (intersection == null || intersection.isEmpty()) {
+ return CrosspointsPaths.EMPTY;
+ }
+ }
+ assert intersection != null;
+ // Limit intersection number to limit crosspoints vertices in result
+ int size = intersection.size();
+ if (limit != NO_LIMIT && size > limit) {
+ intersection = newList(intersection).subList(0, size - 1);
+ }
+
+ // Filter intersection paths
+ List<Path> results = newList();
+ for (Path path : paths) {
+ List<Id> vertices = path.vertices();
+ int length = vertices.size();
+ if (intersection.contains(vertices.get(length - 1))) {
Review Comment:
I will check it later（merge it first）
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291240110
##########
.gitignore:
##########
@@ -7,6 +7,8 @@ upload-files/
demo*
gen-java
*.class
+swagger-ui-*
+hugegraph-dist/dist.sh
Review Comment:
ok, fixed and rollback change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] imbajin commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291153745
##########
.gitignore:
##########
@@ -7,6 +7,8 @@ upload-files/
demo*
gen-java
*.class
+swagger-ui-*
+hugegraph-dist/dist.sh
Review Comment:
Thanks, but we need fix/remove it in the `dist-script` rather than exclude them in git (will be fixed it in another PR) @VGalaxies
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] imbajin commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291153745
##########
.gitignore:
##########
@@ -7,6 +7,8 @@ upload-files/
demo*
gen-java
*.class
+swagger-ui-*
+hugegraph-dist/dist.sh
Review Comment:
Thanks, but we need fix it in the dist-script rather than exclude them in git (will be fixed it in another PR) @VGalaxies
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292234817
##########
hugegraph-api/src/main/java/org/apache/hugegraph/config/ServerOptions.java:
##########
@@ -256,12 +218,24 @@ public static synchronized ServerOptions instance() {
null,
"hugegraph"
);
-
public static final ConfigOption<Boolean> ENABLE_DYNAMIC_CREATE_DROP =
new ConfigOption<>(
"graphs.enable_dynamic_create_drop",
"Whether to enable create or drop graph dynamically.",
disallowEmpty(),
true
);
+ private static volatile ServerOptions instance;
+
+ private ServerOptions() {
+ super();
+ }
+
+ public static synchronized ServerOptions instance() {
+ if (instance == null) {
+ instance = new ServerOptions();
+ instance.registerOptions();
+ }
+ return instance;
+ }
Review Comment:
fixed
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/records/SingleWayMultiPathsRecords.java:
##########
@@ -44,7 +45,7 @@ public abstract class SingleWayMultiPathsRecords extends AbstractRecords {
private final int sourceCode;
private final boolean nearest;
private final IntSet accessedVertices;
-
+ private final HugeTraverser.EdgeRecord edgeRecord;
Review Comment:
fixed
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/records/SingleWayMultiPathsRecords.java:
##########
@@ -176,6 +178,10 @@ protected final Stack<Record> records() {
return this.records;
}
+ public HugeTraverser.EdgeRecord getEdgeIdRecord() {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292235065
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/ShortestPathTraverser.java:
##########
@@ -126,27 +135,36 @@ public PathSet allShortestPaths(Id sourceV, Id targetV, Directions dir,
checkCapacity(traverser.capacity, traverser.accessed(),
"shortest path");
}
+
+ this.vertexIterCounter.addAndGet(traverser.vertexCount);
+ this.edgeIterCounter.addAndGet(traverser.record.accessed());
+
+ paths.setEdges(traverser.edgeRecord.getEdges(paths));
return paths;
}
private class Traverser {
private final ShortestPathRecords record;
+ private final EdgeRecord edgeRecord;
Review Comment:
fixed
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CustomizePathsTraverser.java:
##########
@@ -133,36 +171,13 @@ public List<Path> customizedPaths(Iterator<Vertex> vertices,
return paths;
}
- public static List<Path> topNPath(List<Path> paths,
- boolean incr, long limit) {
- paths.sort((p1, p2) -> {
- WeightPath wp1 = (WeightPath) p1;
- WeightPath wp2 = (WeightPath) p2;
- int result = Double.compare(wp1.totalWeight(), wp2.totalWeight());
- return incr ? result : -result;
- });
-
- if (limit == NO_LIMIT || paths.size() <= limit) {
- return paths;
- }
- return paths.subList(0, (int) limit);
- }
-
- private static List<Node> sample(List<Node> nodes, long sample) {
- if (nodes.size() <= sample) {
- return nodes;
- }
- List<Node> result = newList((int) sample);
- int size = nodes.size();
- for (int random : CollectionUtil.randomSet(0, size, (int) sample)) {
- result.add(nodes.get(random));
- }
- return result;
+ public EdgeRecord getEdgeRecord() {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291009910
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/HugeTraverser.java:
##########
@@ -82,13 +82,16 @@ public class HugeTraverser {
public static final String DEFAULT_SAMPLE = "100";
public static final String DEFAULT_WEIGHT = "0";
public static final int DEFAULT_MAX_DEPTH = 5000;
-
- protected static final int MAX_VERTICES = 10;
-
// Empirical value of scan limit, with which results can be returned in 3s
public static final String DEFAULT_PAGE_LIMIT = "100000";
-
public static final long NO_LIMIT = -1L;
+ protected static final Logger LOG = Log.logger(HugeTraverser.class);
+ protected static final int MAX_VERTICES = 10;
+ private static CollectionFactory collectionFactory;
+ private final HugeGraph graph;
+ // for apimeasure
+ public AtomicLong edgeIterCounter = new AtomicLong(0);
Review Comment:
HugeTraverser is the root class of all traverser algorithms, and there is a need for all sub-traverser algorithms to count the measuring statistics during traversering. So I think of two ways to solve this problems:
1. Give HugeTraverser the ability to track the measuring statistics;
2. Track the measuing statistics in all traverser functions which the Traverser APIs would call
I think the first way is better & more elegant and this is the current implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] codecov[bot] commented on pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#issuecomment-1655365809
## [Codecov](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#2262](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (c7825a6) into [master](https://app.codecov.io/gh/apache/incubator-hugegraph/commit/b02c2bdaa7c88ee0dd3dd89f8e8beb386c24ed17?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (b02c2bd) will **decrease** coverage by `19.70%`.
> The diff coverage is `64.30%`.
```diff
@@ Coverage Diff @@
## master #2262 +/- ##
=============================================
- Coverage 68.63% 48.93% -19.70%
+ Complexity 977 654 -323
=============================================
Files 498 498
Lines 40684 41189 +505
Branches 5681 5713 +32
=============================================
- Hits 27922 20156 -7766
- Misses 10056 18870 +8814
+ Partials 2706 2163 -543
```
| [Files Changed](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...e/hugegraph/api/traversers/CustomizedPathsAPI.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWFwaS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS90cmF2ZXJzZXJzL0N1c3RvbWl6ZWRQYXRoc0FQSS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/hugegraph/api/traversers/EdgesAPI.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWFwaS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS90cmF2ZXJzZXJzL0VkZ2VzQVBJLmphdmE=) | `87.50% <ø> (ø)` | |
| [...g/apache/hugegraph/api/traversers/VerticesAPI.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWFwaS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS90cmF2ZXJzZXJzL1ZlcnRpY2VzQVBJLmphdmE=) | `0.00% <ø> (ø)` | |
| [...aph/job/algorithm/comm/TriangleCountAlgorithm.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1Z2VncmFwaC9qb2IvYWxnb3JpdGhtL2NvbW0vVHJpYW5nbGVDb3VudEFsZ29yaXRobS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...h/traversal/algorithm/CustomizePathsTraverser.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1Z2VncmFwaC90cmF2ZXJzYWwvYWxnb3JpdGhtL0N1c3RvbWl6ZVBhdGhzVHJhdmVyc2VyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...aversal/algorithm/records/ShortestPathRecords.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1Z2VncmFwaC90cmF2ZXJzYWwvYWxnb3JpdGhtL3JlY29yZHMvU2hvcnRlc3RQYXRoUmVjb3Jkcy5qYXZh) | `90.69% <ø> (ø)` | |
| [...che/hugegraph/api/traversers/SameNeighborsAPI.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWFwaS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS90cmF2ZXJzZXJzL1NhbWVOZWlnaGJvcnNBUEkuamF2YQ==) | `30.61% <12.82%> (-69.39%)` | :arrow_down: |
| [...aph/traversal/algorithm/SameNeighborTraverser.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1Z2VncmFwaC90cmF2ZXJzYWwvYWxnb3JpdGhtL1NhbWVOZWlnaGJvclRyYXZlcnNlci5qYXZh) | `41.30% <20.68%> (-52.82%)` | :arrow_down: |
| [...a/org/apache/hugegraph/api/traversers/KoutAPI.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWFwaS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS90cmF2ZXJzZXJzL0tvdXRBUEkuamF2YQ==) | `68.35% <55.00%> (-6.65%)` | :arrow_down: |
| [.../apache/hugegraph/api/traversers/KneighborAPI.java](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVnZWdyYXBoLWFwaS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS90cmF2ZXJzZXJzL0tuZWlnaGJvckFQSS5qYXZh) | `67.56% <55.55%> (-6.95%)` | :arrow_down: |
| ... and [33 more](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
... and [211 files with indirect coverage changes](https://app.codecov.io/gh/apache/incubator-hugegraph/pull/2262/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1277210356
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/SameNeighborTraverser.java:
##########
@@ -46,15 +48,60 @@
Id labelId = this.getEdgeLabelId(label);
Set<Id> sourceNeighbors = IteratorUtils.set(this.adjacentVertices(
- vertex, direction, labelId, degree));
+ vertex, direction, labelId, degree));
Set<Id> targetNeighbors = IteratorUtils.set(this.adjacentVertices(
- other, direction, labelId, degree));
+ other, direction, labelId, degree));
Set<Id> sameNeighbors = (Set<Id>) CollectionUtil.intersect(
- sourceNeighbors, targetNeighbors);
+ sourceNeighbors, targetNeighbors);
+
+ this.vertexIterCounter.addAndGet(2L);
+ this.edgeIterCounter.addAndGet(sourceNeighbors.size());
+ this.edgeIterCounter.addAndGet(targetNeighbors.size());
+
if (limit != NO_LIMIT) {
int end = Math.min(sameNeighbors.size(), limit);
sameNeighbors = CollectionUtil.subSet(sameNeighbors, 0, end);
}
return sameNeighbors;
}
+
+ public Set<Id> sameNeighbors(List<Id> vertexIds, Directions direction,
+ List<String> labels, long degree, long limit) {
+ E.checkNotNull(vertexIds, "vertex ids");
+ E.checkArgument(vertexIds.size() >= 2, "vertex_list size can't " +
+ "be less than 2");
+ for (Id id : vertexIds) {
+ this.checkVertexExist(id, "vertex");
+ }
+ E.checkNotNull(direction, "direction");
+ checkDegree(degree);
+ checkLimit(limit);
+
+ List<Id> labelsId = new ArrayList<>();
+ if (labels != null) {
+ for (String label : labels) {
+ labelsId.add(this.getEdgeLabelId(label));
+ }
+ }
+
+ Set<Id> sameNeighbors = new HashSet<>();
+ for (int i = 0; i < vertexIds.size(); i++) {
+ Set<Id> vertexNeighbors = IteratorUtils.set(this.adjacentVertices(
+ vertexIds.get(i), direction, labelsId, degree));
+ if (i == 0) {
+ sameNeighbors = vertexNeighbors;
+ } else {
+ sameNeighbors = (Set<Id>) CollectionUtil.intersect(
+ sameNeighbors, vertexNeighbors);
+ }
+ this.vertexIterCounter.addAndGet(1L);
+ this.edgeIterCounter.addAndGet(vertexNeighbors.size());
+ }
+
+ if (limit != NO_LIMIT) {
+ int end = Math.min(sameNeighbors.size(), (int) limit);
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] javeme commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "javeme (via GitHub)" <gi...@apache.org>.
javeme commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1278570485
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
Review Comment:
prefer rename to ApiMeasurer and expect an blank line after a class define
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/TemplatePathsTraverser.java:
##########
@@ -276,4 +280,26 @@ public boolean lastSuperStep() {
this.targetIndex == this.sourceIndex + 1;
}
}
+
+ public static class WrappedPathSet {
Review Comment:
ditto
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
+ public static final String EDGE_ITER = "edge_iters";
+ public static final String VERTICE_ITER = "vertice_iters";
+ public static final String COST = "cost";
+ protected long timeStart = System.currentTimeMillis();
+ protected HashMap<String, Object> mapResult = new LinkedHashMap<>();
+
+ public Map<String, Object> getResult() {
+ mapResult.put(COST, System.currentTimeMillis() - timeStart);
+ return mapResult;
+ }
+
+ public void put(String key, String value) {
+ this.mapResult.put(key, value);
+ }
+
+ public void put(String key, long value) {
+ this.mapResult.put(key, value);
+ }
+
+ public void put(String key, int value) {
+ this.mapResult.put(key, value);
+ }
+
+ protected void addCount(String key, long value) {
+ long cur = 0;
Review Comment:
prefer current
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KneighborAPI.java:
##########
@@ -94,8 +100,14 @@ public String get(@Context GraphManager manager,
try (KneighborTraverser traverser = new KneighborTraverser(g)) {
ids = traverser.kneighbor(source, dir, edgeLabel,
depth, maxDegree, limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+ }
+ if (countOnly) {
+ return manager.serializer(g, measure.getResult()).writeMap(
Review Comment:
prefer to wrap line before ".writeMap"
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
+ public static final String EDGE_ITER = "edge_iters";
+ public static final String VERTICE_ITER = "vertice_iters";
+ public static final String COST = "cost";
+ protected long timeStart = System.currentTimeMillis();
+ protected HashMap<String, Object> mapResult = new LinkedHashMap<>();
Review Comment:
mark private as possible and assign in the constructor
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
+ public static final String EDGE_ITER = "edge_iters";
+ public static final String VERTICE_ITER = "vertice_iters";
+ public static final String COST = "cost";
+ protected long timeStart = System.currentTimeMillis();
+ protected HashMap<String, Object> mapResult = new LinkedHashMap<>();
+
+ public Map<String, Object> getResult() {
+ mapResult.put(COST, System.currentTimeMillis() - timeStart);
+ return mapResult;
+ }
+
+ public void put(String key, String value) {
+ this.mapResult.put(key, value);
+ }
+
+ public void put(String key, long value) {
+ this.mapResult.put(key, value);
+ }
+
+ public void put(String key, int value) {
+ this.mapResult.put(key, value);
+ }
+
+ protected void addCount(String key, long value) {
+ long cur = 0;
+ if (this.mapResult.containsKey(key)) {
+ cur = (long) this.mapResult.get(key);
+ }
+ this.mapResult.put(key, cur + value);
Review Comment:
can we use a MutableLong instead? then we don't need to put map every time
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/SingleSourceShortestPathTraverser.java:
##########
@@ -91,34 +102,129 @@ public NodeWithWeight weightedShortestPath(Id sourceV, Id targetV,
traverser.forward();
Map<Id, NodeWithWeight> results = traverser.shortestPaths();
if (results.containsKey(targetV) || traverser.done()) {
- return results.get(targetV);
+ this.vertexIterCounter.addAndGet(traverser.vertexCount);
+ this.edgeIterCounter.addAndGet(traverser.edgeCount);
+ NodeWithWeight nodeWithWeight = results.get(targetV);
+ if (nodeWithWeight != null) {
+ Iterator<Id> vertexIter = nodeWithWeight.node.path().iterator();
+ Set<Edge> edges = traverser.edgeRecord.getEdges(vertexIter);
+ nodeWithWeight.setEdges(edges);
+ }
+ return nodeWithWeight;
}
checkCapacity(traverser.capacity, traverser.size, "shortest path");
}
}
+ public static class NodeWithWeight implements Comparable<NodeWithWeight> {
+
+ private final double weight;
+ private final Node node;
+
+ private Set<Edge> edges = Collections.emptySet();
+
+ public NodeWithWeight(double weight, Node node) {
+ this.weight = weight;
+ this.node = node;
+ }
+
+ public NodeWithWeight(double weight, Id id, NodeWithWeight prio) {
+ this(weight, new Node(id, prio.node()));
+ }
+
+ public Set<Edge> getEdges() {
+ return edges;
+ }
+
+ public void setEdges(Set<Edge> edges) {
+ this.edges = edges;
+ }
+
+ public double weight() {
+ return weight;
+ }
+
+ public Node node() {
+ return this.node;
+ }
+
+ public Map<String, Object> toMap() {
+ return ImmutableMap.of("weight", this.weight,
+ "vertices", this.node().path());
+ }
+
+ @Override
+ public int compareTo(NodeWithWeight other) {
+ return Double.compare(this.weight, other.weight);
+ }
+ }
+
+ public static class WeightedPaths extends LinkedHashMap<Id, NodeWithWeight> {
+
+ private static final long serialVersionUID = -313873642177730993L;
+ private Set<Edge> edges = Collections.emptySet();
+
+ public Set<Edge> getEdges() {
+ return edges;
+ }
+
+ public void setEdges(Set<Edge> edges) {
+ this.edges = edges;
+ }
Review Comment:
can keep "edges()" name style, please also update other places
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
+ public static final String EDGE_ITER = "edge_iters";
Review Comment:
rename to edge_iterations?
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
+ public static final String EDGE_ITER = "edge_iters";
+ public static final String VERTICE_ITER = "vertice_iters";
+ public static final String COST = "cost";
+ protected long timeStart = System.currentTimeMillis();
+ protected HashMap<String, Object> mapResult = new LinkedHashMap<>();
+
+ public Map<String, Object> getResult() {
Review Comment:
prefer `measures()` name
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
+ public static final String EDGE_ITER = "edge_iters";
+ public static final String VERTICE_ITER = "vertice_iters";
+ public static final String COST = "cost";
+ protected long timeStart = System.currentTimeMillis();
+ protected HashMap<String, Object> mapResult = new LinkedHashMap<>();
Review Comment:
rename to measures? and we can use InsertionOrderUtil.newMap()
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CollectionPathsTraverser.java:
##########
@@ -274,4 +280,26 @@ protected int accessedNodes() {
return this.sourcesAll.size() + this.targetsAll.size();
}
}
+
+ public static class WrappedPathCollection {
Review Comment:
ditto
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/MultiNodeShortestPathTraverser.java:
##########
@@ -56,70 +83,69 @@ public List<Path> multiNodeShortestPath(Iterator<Vertex> vertices,
});
if (maxDepth >= this.concurrentDepth() && vertexCount > 10) {
- return this.multiNodeShortestPathConcurrent(pairs, step,
- maxDepth, capacity);
+ return this.multiNodeShortestPathConcurrent(pairs, step, maxDepth, capacity);
} else {
- return this.multiNodeShortestPathSingle(pairs, step,
- maxDepth, capacity);
+ return this.multiNodeShortestPathSingle(pairs, step, maxDepth, capacity);
}
}
- public List<Path> multiNodeShortestPathConcurrent(List<Pair<Id, Id>> pairs,
- EdgeStep step,
- int maxDepth,
- long capacity) {
- List<Path> results = new CopyOnWriteArrayList<>();
+ public WrappedListPath multiNodeShortestPathConcurrent(List<Pair<Id, Id>> pairs,
+ EdgeStep step, int maxDepth,
+ long capacity) {
+ List<Path> paths = new CopyOnWriteArrayList<>();
+ Set<Edge> edges = new CopyOnWriteArraySet<>();
ShortestPathTraverser traverser =
- new ShortestPathTraverser(this.graph());
+ new ShortestPathTraverser(this.graph());
this.traversePairs(pairs.iterator(), pair -> {
Path path = traverser.shortestPath(pair.getLeft(), pair.getRight(),
step, maxDepth, capacity);
if (!Path.EMPTY.equals(path)) {
- results.add(path);
+ paths.add(path);
}
+ edges.addAll(path.getEdges());
});
+ this.vertexIterCounter.addAndGet(traverser.vertexIterCounter.get());
+ this.edgeIterCounter.addAndGet(traverser.edgeIterCounter.get());
- return results;
+ return new WrappedListPath(paths, edges);
}
- public List<Path> multiNodeShortestPathSingle(List<Pair<Id, Id>> pairs,
- EdgeStep step, int maxDepth,
- long capacity) {
- List<Path> results = newList();
+ public WrappedListPath multiNodeShortestPathSingle(List<Pair<Id, Id>> pairs,
+ EdgeStep step, int maxDepth,
+ long capacity) {
+ List<Path> paths = newList();
+ Set<Edge> edges = newSet();
ShortestPathTraverser traverser =
- new ShortestPathTraverser(this.graph());
+ new ShortestPathTraverser(this.graph());
for (Pair<Id, Id> pair : pairs) {
Path path = traverser.shortestPath(pair.getLeft(), pair.getRight(),
step, maxDepth, capacity);
if (!Path.EMPTY.equals(path)) {
- results.add(path);
+ paths.add(path);
}
+ edges.addAll(path.getEdges());
}
- return results;
+ this.vertexIterCounter.addAndGet(traverser.vertexIterCounter.get());
+ this.edgeIterCounter.addAndGet(traverser.edgeIterCounter.get());
+
+ return new WrappedListPath(paths, edges);
}
- private static <T> void cmn(List<T> all, int m, int n, int current,
- List<T> result, Consumer<List<T>> consumer) {
- assert m <= all.size();
- assert current <= all.size();
- if (result == null) {
- result = newList(n);
- }
- if (n == 0) {
- // All n items are selected
- consumer.accept(result);
- return;
+ public static class WrappedListPath {
Review Comment:
ditto
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/TemplatePathsTraverser.java:
##########
@@ -276,4 +280,26 @@ public boolean lastSuperStep() {
this.targetIndex == this.sourceIndex + 1;
}
}
+
+ public static class WrappedPathSet {
+ private final Set<Path> paths;
+ private Set<Edge> edges = Collections.emptySet();
+
+ public WrappedPathSet(Set<Path> paths) {
+ this.paths = paths;
+ }
+
+ public WrappedPathSet(Set<Path> paths, Set<Edge> edges) {
+ this.paths = paths;
+ this.edges = edges;
+ }
+
+ public Set<Path> getPaths() {
+ return paths;
+ }
+
+ public Set<Edge> getEdges() {
+ return edges;
+ }
Review Comment:
ditto
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/AllShortestPathsAPI.java:
##########
@@ -86,8 +97,34 @@ public String get(@Context GraphManager manager,
List<String> edgeLabels = edgeLabel == null ? ImmutableList.of() :
ImmutableList.of(edgeLabel);
HugeTraverser.PathSet paths = traverser.allShortestPaths(
- sourceId, targetId, dir, edgeLabels,
- depth, maxDegree, skipDegree, capacity);
- return manager.serializer(g).writePaths("paths", paths, false);
+ sourceId, targetId, dir, edgeLabels,
Review Comment:
prefer keep the old style
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/CustomizedCrosspointsAPI.java:
##########
@@ -22,45 +22,62 @@
import static org.apache.hugegraph.traversal.algorithm.HugeTraverser.DEFAULT_PATHS_LIMIT;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
-import io.swagger.v3.oas.annotations.tags.Tag;
-import jakarta.inject.Singleton;
-import jakarta.ws.rs.Consumes;
-import jakarta.ws.rs.POST;
-import jakarta.ws.rs.Path;
-import jakarta.ws.rs.PathParam;
-import jakarta.ws.rs.Produces;
-import jakarta.ws.rs.core.Context;
-
-import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.core.GraphManager;
-import org.slf4j.Logger;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.api.API;
import org.apache.hugegraph.backend.id.Id;
-import org.apache.hugegraph.backend.query.QueryResults;
+import org.apache.hugegraph.core.GraphManager;
import org.apache.hugegraph.traversal.algorithm.CustomizedCrosspointsTraverser;
import org.apache.hugegraph.traversal.algorithm.HugeTraverser;
import org.apache.hugegraph.type.define.Directions;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.slf4j.Logger;
+
import com.codahale.metrics.annotation.Timed;
import com.fasterxml.jackson.annotation.JsonAlias;
import com.fasterxml.jackson.annotation.JsonProperty;
+import io.swagger.v3.oas.annotations.tags.Tag;
+import jakarta.inject.Singleton;
+import jakarta.ws.rs.Consumes;
+import jakarta.ws.rs.POST;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.PathParam;
+import jakarta.ws.rs.Produces;
+import jakarta.ws.rs.core.Context;
+
@Path("graphs/{graph}/traversers/customizedcrosspoints")
@Singleton
@Tag(name = "CustomizedCrosspointsAPI")
public class CustomizedCrosspointsAPI extends API {
private static final Logger LOG = Log.logger(CustomizedCrosspointsAPI.class);
+ private static List<CustomizedCrosspointsTraverser.PathPattern> pathPatterns(
+ HugeGraph graph, CrosspointsRequest request) {
+ int stepSize = request.pathPatterns.size();
+ List<CustomizedCrosspointsTraverser.PathPattern> pathPatterns;
+ pathPatterns = new ArrayList<>(stepSize);
+ for (PathPattern pattern : request.pathPatterns) {
+ CustomizedCrosspointsTraverser.PathPattern pathPattern;
+ pathPattern = new CustomizedCrosspointsTraverser.PathPattern();
Review Comment:
ditto
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/CustomizedCrosspointsAPI.java:
##########
@@ -22,45 +22,62 @@
import static org.apache.hugegraph.traversal.algorithm.HugeTraverser.DEFAULT_PATHS_LIMIT;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
-import io.swagger.v3.oas.annotations.tags.Tag;
-import jakarta.inject.Singleton;
-import jakarta.ws.rs.Consumes;
-import jakarta.ws.rs.POST;
-import jakarta.ws.rs.Path;
-import jakarta.ws.rs.PathParam;
-import jakarta.ws.rs.Produces;
-import jakarta.ws.rs.core.Context;
-
-import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.core.GraphManager;
-import org.slf4j.Logger;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.api.API;
import org.apache.hugegraph.backend.id.Id;
-import org.apache.hugegraph.backend.query.QueryResults;
+import org.apache.hugegraph.core.GraphManager;
import org.apache.hugegraph.traversal.algorithm.CustomizedCrosspointsTraverser;
import org.apache.hugegraph.traversal.algorithm.HugeTraverser;
import org.apache.hugegraph.type.define.Directions;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.slf4j.Logger;
+
import com.codahale.metrics.annotation.Timed;
import com.fasterxml.jackson.annotation.JsonAlias;
import com.fasterxml.jackson.annotation.JsonProperty;
+import io.swagger.v3.oas.annotations.tags.Tag;
+import jakarta.inject.Singleton;
+import jakarta.ws.rs.Consumes;
+import jakarta.ws.rs.POST;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.PathParam;
+import jakarta.ws.rs.Produces;
+import jakarta.ws.rs.core.Context;
+
@Path("graphs/{graph}/traversers/customizedcrosspoints")
@Singleton
@Tag(name = "CustomizedCrosspointsAPI")
public class CustomizedCrosspointsAPI extends API {
private static final Logger LOG = Log.logger(CustomizedCrosspointsAPI.class);
+ private static List<CustomizedCrosspointsTraverser.PathPattern> pathPatterns(
+ HugeGraph graph, CrosspointsRequest request) {
+ int stepSize = request.pathPatterns.size();
+ List<CustomizedCrosspointsTraverser.PathPattern> pathPatterns;
+ pathPatterns = new ArrayList<>(stepSize);
Review Comment:
one line is ok
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/CrosspointsAPI.java:
##########
@@ -84,6 +85,9 @@ public String get(@Context GraphManager manager,
dir, edgeLabel, depth,
maxDegree, capacity,
limit);
- return manager.serializer(g).writePaths("crosspoints", paths, true);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+ return manager.serializer(g, measure.getResult())
Review Comment:
prefer to update the `serializer(g)` method and set default an ApiMeasure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1278759670
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -190,4 +192,42 @@ public static boolean checkAndParseAction(String action) {
String.format("Not support action '%s'", action));
}
}
+
+ public static class ApiMeasure {
+ public static final String EDGE_ITER = "edge_iters";
+ public static final String VERTICE_ITER = "vertice_iters";
+ public static final String COST = "cost";
+ protected long timeStart = System.currentTimeMillis();
+ protected HashMap<String, Object> mapResult = new LinkedHashMap<>();
+
+ public Map<String, Object> getResult() {
+ mapResult.put(COST, System.currentTimeMillis() - timeStart);
+ return mapResult;
+ }
+
+ public void put(String key, String value) {
+ this.mapResult.put(key, value);
+ }
+
+ public void put(String key, long value) {
+ this.mapResult.put(key, value);
+ }
+
+ public void put(String key, int value) {
+ this.mapResult.put(key, value);
+ }
+
+ protected void addCount(String key, long value) {
+ long cur = 0;
+ if (this.mapResult.containsKey(key)) {
+ cur = (long) this.mapResult.get(key);
+ }
+ this.mapResult.put(key, cur + value);
Review Comment:
fixed
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/CollectionPathsTraverser.java:
##########
@@ -274,4 +280,26 @@ protected int accessedNodes() {
return this.sourcesAll.size() + this.targetsAll.size();
}
}
+
+ public static class WrappedPathCollection {
Review Comment:
fixed
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/MultiNodeShortestPathTraverser.java:
##########
@@ -56,70 +83,69 @@ public List<Path> multiNodeShortestPath(Iterator<Vertex> vertices,
});
if (maxDepth >= this.concurrentDepth() && vertexCount > 10) {
- return this.multiNodeShortestPathConcurrent(pairs, step,
- maxDepth, capacity);
+ return this.multiNodeShortestPathConcurrent(pairs, step, maxDepth, capacity);
} else {
- return this.multiNodeShortestPathSingle(pairs, step,
- maxDepth, capacity);
+ return this.multiNodeShortestPathSingle(pairs, step, maxDepth, capacity);
}
}
- public List<Path> multiNodeShortestPathConcurrent(List<Pair<Id, Id>> pairs,
- EdgeStep step,
- int maxDepth,
- long capacity) {
- List<Path> results = new CopyOnWriteArrayList<>();
+ public WrappedListPath multiNodeShortestPathConcurrent(List<Pair<Id, Id>> pairs,
+ EdgeStep step, int maxDepth,
+ long capacity) {
+ List<Path> paths = new CopyOnWriteArrayList<>();
+ Set<Edge> edges = new CopyOnWriteArraySet<>();
ShortestPathTraverser traverser =
- new ShortestPathTraverser(this.graph());
+ new ShortestPathTraverser(this.graph());
this.traversePairs(pairs.iterator(), pair -> {
Path path = traverser.shortestPath(pair.getLeft(), pair.getRight(),
step, maxDepth, capacity);
if (!Path.EMPTY.equals(path)) {
- results.add(path);
+ paths.add(path);
}
+ edges.addAll(path.getEdges());
});
+ this.vertexIterCounter.addAndGet(traverser.vertexIterCounter.get());
+ this.edgeIterCounter.addAndGet(traverser.edgeIterCounter.get());
- return results;
+ return new WrappedListPath(paths, edges);
}
- public List<Path> multiNodeShortestPathSingle(List<Pair<Id, Id>> pairs,
- EdgeStep step, int maxDepth,
- long capacity) {
- List<Path> results = newList();
+ public WrappedListPath multiNodeShortestPathSingle(List<Pair<Id, Id>> pairs,
+ EdgeStep step, int maxDepth,
+ long capacity) {
+ List<Path> paths = newList();
+ Set<Edge> edges = newSet();
ShortestPathTraverser traverser =
- new ShortestPathTraverser(this.graph());
+ new ShortestPathTraverser(this.graph());
for (Pair<Id, Id> pair : pairs) {
Path path = traverser.shortestPath(pair.getLeft(), pair.getRight(),
step, maxDepth, capacity);
if (!Path.EMPTY.equals(path)) {
- results.add(path);
+ paths.add(path);
}
+ edges.addAll(path.getEdges());
}
- return results;
+ this.vertexIterCounter.addAndGet(traverser.vertexIterCounter.get());
+ this.edgeIterCounter.addAndGet(traverser.edgeIterCounter.get());
+
+ return new WrappedListPath(paths, edges);
}
- private static <T> void cmn(List<T> all, int m, int n, int current,
- List<T> result, Consumer<List<T>> consumer) {
- assert m <= all.size();
- assert current <= all.size();
- if (result == null) {
- result = newList(n);
- }
- if (n == 0) {
- // All n items are selected
- consumer.accept(result);
- return;
+ public static class WrappedListPath {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292234767
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KoutAPI.java:
##########
@@ -116,32 +129,35 @@ public String post(@Context GraphManager manager,
E.checkArgument(request.step != null,
"The steps of request can't be null");
if (request.countOnly) {
- E.checkArgument(!request.withVertex && !request.withPath,
- "Can't return vertex or path when count only");
+ E.checkArgument(!request.withVertex && !request.withPath && !request.withEdge,
+ "Can't return vertex or path or edge when count only");
Review Comment:
fixed
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/KoutAPI.java:
##########
@@ -99,8 +105,15 @@ public String get(@Context GraphManager manager,
try (KoutTraverser traverser = new KoutTraverser(g)) {
ids = traverser.kout(sourceId, dir, edgeLabel, depth,
nearest, maxDegree, capacity, limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+ }
+
+ if (count_only) {
+ return manager.serializer(g, measure.measures()).writeMap(ImmutableMap.of(
Review Comment:
fixed
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/SameNeighborsAPI.java:
##########
@@ -77,6 +89,81 @@ public String get(@Context GraphManager manager,
SameNeighborTraverser traverser = new SameNeighborTraverser(g);
Set<Id> neighbors = traverser.sameNeighbors(sourceId, targetId, dir,
edgeLabel, maxDegree, limit);
- return manager.serializer(g).writeList("same_neighbors", neighbors);
+
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ return manager.serializer(g, measure.measures())
+ .writeList("same_neighbors", neighbors);
+ }
+
+ @POST
+ @Timed
+ @Produces(APPLICATION_JSON_WITH_CHARSET)
+ public String sameNeighbors(@Context GraphManager manager,
+ @PathParam("graph") String graph,
+ Request request) {
+ LOG.debug("Graph [{}] get same neighbors among batch, '{}'", graph, request.toString());
+
+ ApiMeasurer measure = new ApiMeasurer();
+
+ Directions dir = Directions.convert(EdgeAPI.parseDirection(request.direction));
+ HugeGraph g = graph(manager, graph);
+ SameNeighborTraverser traverser = new SameNeighborTraverser(g);
+
+ List<Object> vertexList = request.vertexList;
+ E.checkArgument(vertexList.size() >= 2, "vertex_list size can't " +
+ "be less than 2");
+
+ List<Id> vertexIds = new ArrayList<>();
+ for (Object obj : vertexList) {
+ vertexIds.add(HugeVertex.getIdValue(obj));
+ }
+
+ Set<Id> neighbors = traverser.sameNeighbors(vertexIds, dir, request.labels,
+ request.maxDegree, request.limit);
+ measure.addIterCount(traverser.vertexIterCounter.get(),
+ traverser.edgeIterCounter.get());
+
+ Iterator<?> iterVertex;
+ Set<Id> ids = new HashSet<>(neighbors);
+ ids.addAll(vertexIds);
+ if (request.withVertex && !ids.isEmpty()) {
+ iterVertex = g.vertices(ids.toArray());
+ } else {
+ iterVertex = ids.iterator();
+ }
+ return manager.serializer(g, measure.measures())
+ .writeMap(ImmutableMap.of("same_neighbors", neighbors,
+ "vertices", iterVertex));
+ }
+
+ private static class Request {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1292234700
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/CustomizedPathsAPI.java:
##########
@@ -30,40 +30,50 @@
import java.util.Map;
import java.util.Set;
-import io.swagger.v3.oas.annotations.tags.Tag;
-import jakarta.inject.Singleton;
-import jakarta.ws.rs.Consumes;
-import jakarta.ws.rs.POST;
-import jakarta.ws.rs.Path;
-import jakarta.ws.rs.PathParam;
-import jakarta.ws.rs.Produces;
-import jakarta.ws.rs.core.Context;
-
-import org.apache.tinkerpop.gremlin.structure.Vertex;
-import org.apache.hugegraph.core.GraphManager;
-import org.slf4j.Logger;
-
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.api.API;
import org.apache.hugegraph.backend.id.Id;
-import org.apache.hugegraph.backend.query.QueryResults;
+import org.apache.hugegraph.core.GraphManager;
import org.apache.hugegraph.traversal.algorithm.CustomizePathsTraverser;
import org.apache.hugegraph.traversal.algorithm.HugeTraverser;
import org.apache.hugegraph.traversal.algorithm.steps.WeightedEdgeStep;
import org.apache.hugegraph.type.define.Directions;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.slf4j.Logger;
+
import com.codahale.metrics.annotation.Timed;
import com.fasterxml.jackson.annotation.JsonAlias;
import com.fasterxml.jackson.annotation.JsonProperty;
+import io.swagger.v3.oas.annotations.tags.Tag;
+import jakarta.inject.Singleton;
+import jakarta.ws.rs.Consumes;
+import jakarta.ws.rs.POST;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.PathParam;
+import jakarta.ws.rs.Produces;
+import jakarta.ws.rs.core.Context;
+
@Path("graphs/{graph}/traversers/customizedpaths")
@Singleton
@Tag(name = "CustomizedPathsAPI")
public class CustomizedPathsAPI extends API {
private static final Logger LOG = Log.logger(CustomizedPathsAPI.class);
+ private static List<WeightedEdgeStep> step(HugeGraph graph,
+ PathRequest req) {
Review Comment:
fixed
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/traversers/FusiformSimilarityAPI.java:
##########
@@ -94,28 +95,38 @@ public String post(@Context GraphManager manager,
request.minNeighbors, request.alpha, request.minSimilars,
request.groupProperty, request.minGroups);
+ ApiMeasurer measure = new ApiMeasurer();
HugeGraph g = graph(manager, graph);
Iterator<Vertex> sources = request.sources.vertices(g);
E.checkArgument(sources != null && sources.hasNext(),
"The source vertices can't be empty");
FusiformSimilarityTraverser traverser =
- new FusiformSimilarityTraverser(g);
+ new FusiformSimilarityTraverser(g);
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] imbajin commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291153745
##########
.gitignore:
##########
@@ -7,6 +7,8 @@ upload-files/
demo*
gen-java
*.class
+swagger-ui-*
+hugegraph-dist/dist.sh
Review Comment:
Thanks, but we need fix/[remove it](https://github.com/apache/incubator-hugegraph/pull/2275/commits/fc31c2e1024d72f91e3ce17972446c43620a7a86) in the `dist-script` rather than exclude them in git (will be fixed it in another PR) @VGalaxies
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291009910
##########
hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/HugeTraverser.java:
##########
@@ -82,13 +82,16 @@ public class HugeTraverser {
public static final String DEFAULT_SAMPLE = "100";
public static final String DEFAULT_WEIGHT = "0";
public static final int DEFAULT_MAX_DEPTH = 5000;
-
- protected static final int MAX_VERTICES = 10;
-
// Empirical value of scan limit, with which results can be returned in 3s
public static final String DEFAULT_PAGE_LIMIT = "100000";
-
public static final long NO_LIMIT = -1L;
+ protected static final Logger LOG = Log.logger(HugeTraverser.class);
+ protected static final int MAX_VERTICES = 10;
+ private static CollectionFactory collectionFactory;
+ private final HugeGraph graph;
+ // for apimeasure
+ public AtomicLong edgeIterCounter = new AtomicLong(0);
Review Comment:
HugeTraverser is the root class of all traverser algorithms, and there is a need for all sub-traverser algorithms to count the measuring statistics during traversering. So I think of two ways to solve this problems:
1. Give HugeTraverser the ability to track the measuring statistics;
2. Track the measuing statistics in all traverser functions which the Traverser APIs would call
I think the first one is better & more elegant and this is the current implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] DanGuge commented on a diff in pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "DanGuge (via GitHub)" <gi...@apache.org>.
DanGuge commented on code in PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262#discussion_r1291011776
##########
hugegraph-api/src/main/java/org/apache/hugegraph/api/API.java:
##########
@@ -186,8 +185,56 @@ public static boolean checkAndParseAction(String action) {
} else if (action.equals(ACTION_ELIMINATE)) {
return false;
} else {
- throw new NotSupportedException(
- String.format("Not support action '%s'", action));
+ throw new NotSupportedException(String.format("Not support action '%s'", action));
+ }
+ }
+
+ public static class ApiMeasurer {
+
+ public static final String EDGE_ITER = "edge_iterations";
+ public static final String VERTICE_ITER = "vertice_iterations";
+ public static final String COST = "cost";
+ private final long timeStart;
+ private final Map<String, Object> measures;
+
+ public ApiMeasurer() {
+ this.timeStart = System.currentTimeMillis();
+ this.measures = InsertionOrderUtil.newMap();
+ }
+
+ public Map<String, Object> measures() {
+ measures.put(COST, System.currentTimeMillis() - timeStart);
+ return measures;
+ }
+
+ public void put(String key, String value) {
+ this.measures.put(key, value);
+ }
+
+ public void put(String key, long value) {
+ this.measures.put(key, value);
+ }
+
+ public void put(String key, int value) {
+ this.measures.put(key, value);
+ }
+
+ protected void addCount(String key, long value) {
+ Object current = measures.get(key);
+ if (current == null) {
+ measures.put(key, new MutableLong(value));
+ } else if (current instanceof MutableLong) {
+ MutableLong currentMutableLong = (MutableLong) current;
+ currentMutableLong.add(value);
+ } else if (current instanceof Long) {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
[GitHub] [incubator-hugegraph] imbajin merged pull request #2262: feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin merged PR #2262:
URL: https://github.com/apache/incubator-hugegraph/pull/2262
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org