You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hugegraph.apache.org by "haohao0103 (via GitHub)" <gi...@apache.org> on 2023/06/14 05:25:58 UTC

[GitHub] [incubator-hugegraph-toolchain] haohao0103 opened a new pull request, #480: fix Spark loader has dependency conflicts

haohao0103 opened a new pull request, #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1261273269


##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/constant/Constants.java:
##########
@@ -81,4 +81,6 @@ public final class Constants {
     public static final String LOAD_DATA_PARSE_SUFFIX = "parse";
     public static final String LOAD_DATA_SER_SUFFIX = "ser";
     public static final String LOAD_DATA_INSERT_SUFFIX = "insert";
+    public static final String ACTION_DELETE = "delete";

Review Comment:
   there is no impact on rocksdb backend's data, because by pass is not implemented for rocksdb backend.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1661407484

   > 你好,@haohao0103,将两个东西分开放在不同的 PR 中会更好吗?
   
   @liuxiaocs7 hi ,I have revert  the last few commits and will deal with it after this pr merge.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] liuxiaocs7 commented on pull request #480: fix: Spark with loader has dependency conflicts

Posted by "liuxiaocs7 (via GitHub)" <gi...@apache.org>.
liuxiaocs7 commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1664247875

   Hi, @haohao0103, thanks for your work, but -1 for me.
   
   I tested locally use `./bin/hugegraph-spark-loader.sh --master local[*] --name spark-hugegraph-loader --file ./example/spark/struct.json --host 192.168.34.164 --port 18080 --graph hugegraph`
   
   but got:
   
   ```
   java.lang.NullPointerException
           at java.lang.String.split(String.java:2337)
           at java.lang.String.split(String.java:2422)
   ```
   
   If I remember correctly, delimieter has a default comma, we need to check this?
   
   ```
   elements = builder.build(fileSource.header(),
                            row.mkString(delimiter).split(delimiter));
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1665352528

   > Hi, @haohao0103, thanks for your work, but -1 for me.
   > 
   > I tested locally use `./bin/hugegraph-spark-loader.sh --master local[*] --name spark-hugegraph-loader --file ./example/spark/struct.json --host 192.168.34.164 --port 18080 --graph hugegraph`
   > 
   > but got:
   > 
   > ```
   > java.lang.NullPointerException
   >         at java.lang.String.split(String.java:2337)
   >         at java.lang.String.split(String.java:2422)
   > ```
   > 
   > If I remember correctly, delimieter has a default comma, we need to check this?
   > 
   > ```
   > elements = builder.build(fileSource.header(),
   >                          row.mkString(delimiter).split(delimiter));
   > ```
   
   hi,i have check it .when file format is json the  default  delimiter is null。
   ![image](https://github.com/apache/incubator-hugegraph-toolchain/assets/30570711/2647a655-27e9-40d1-a93d-099fbc78747a)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] javeme commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "javeme (via GitHub)" <gi...@apache.org>.
javeme commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1261160828


##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/constant/Constants.java:
##########
@@ -81,4 +81,6 @@ public final class Constants {
     public static final String LOAD_DATA_PARSE_SUFFIX = "parse";
     public static final String LOAD_DATA_SER_SUFFIX = "ser";
     public static final String LOAD_DATA_INSERT_SUFFIX = "insert";
+    public static final String ACTION_DELETE = "delete";

Review Comment:
   what happens if this 'delete' is passed for rocksdb backend?



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/executor/LoadOptions.java:
##########
@@ -217,22 +217,26 @@ public class LoadOptions implements Serializable {
                description = "The number of partitions of the HBase vertex table")
     public int vertexPartitions = 64;
 
-    @Parameter(names = {"edgeTablename"}, arity = 1,
-               description = "edgeTablename")
+    @Parameter(names = {"--edge-table-name"}, arity = 1,
+               description = "The name of the the HBase edge table")
     public String edgeTablename;
-    @Parameter(names = {"vertexTablename"}, arity = 1,
-               description = "vertexTablename")
+    @Parameter(names = {"--vertex-table-name"}, arity = 1,
+               description = "The name of the the HBase vertex table")
     public String vertexTablename;
-    @Parameter(names = {"hbaseZKQuorum"}, arity = 1,
-               description = "hbaseZKQuorum")
+    @Parameter(names = {"--zk-quorum"}, arity = 1,
+               description = "Address of the zookeeper on which hbase depends")
     public String hbaseZKQuorum;
-    @Parameter(names = {"hbaseZKPort"}, arity = 1,
-               description = "hbaseZKPort")
+    @Parameter(names = {"--zk-port"}, arity = 1,
+               description = "port of the zookeeper on which hbase depends")
     public String hbaseZKPort;
-    @Parameter(names = {"hbaseZKParent"}, arity = 1,
-               description = "hbaseZKParent")
+    @Parameter(names = {"--zk-parent"}, arity = 1,
+               description = "hbase root node on the zookeeper")
     public String hbaseZKParent;
 
+    @Parameter(names = {"--action-type"},  arity = 1,
+               description = "The type of operation performed on the graph,update or delete")

Review Comment:
   expect a space "graph,"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] imbajin commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1285078528


##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java:
##########
@@ -292,9 +293,13 @@ private void parse(Row row, Map.Entry<ElementBuilder, List<GraphElement>> builde
             case FILE:
             case HDFS:
                 FileSource fileSource = struct.input().asFileSource();
-                String delimiter = fileSource.delimiter();
-                elements = builder.build(fileSource.header(),
-                                         row.mkString(delimiter).split(delimiter));
+                String delimiter = fileSource.delimiter() ;
+                if (Optional.ofNullable(delimiter).isPresent()) {
+                    elements = builder.build(fileSource.header(),
+                               row.mkString(delimiter).split(delimiter));

Review Comment:
   @liuxiaocs7 could test it again?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] liuxiaocs7 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "liuxiaocs7 (via GitHub)" <gi...@apache.org>.
liuxiaocs7 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1283451850


##########
hugegraph-loader/src/main/resources/META-INF/services/shaded.jakarta.ws.rs.client.ClientBuilder:
##########
@@ -0,0 +1,16 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+shaded.org.glassfish.jersey.client.JerseyClientBuilder

Review Comment:
   I have a question, why do I need to add SPI to these two classes (this one and below), it seems to be able to import normally without adding these two?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] javeme commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "javeme (via GitHub)" <gi...@apache.org>.
javeme commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1258614628


##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/executor/LoadOptions.java:
##########
@@ -233,6 +233,10 @@ public class LoadOptions implements Serializable {
                description = "hbaseZKParent")
     public String hbaseZKParent;
 
+    @Parameter(names = {"--action-type"},  arity = 1,
+            description = "add/update or delete")

Review Comment:
   align with "names"



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/direct/loader/HBaseDirectLoader.java:
##########
@@ -200,23 +201,29 @@ List<Tuple2<ImmutableBytesWritable, KeyValue>> buildAndSer(HBaseSerializer seria
                                                            struct.input().type()));
             }
 
-            boolean isVertex = builder.mapping().type().isVertex();
-            if (isVertex) {
-                for (Vertex vertex : (List<Vertex>) (Object) elementsElement) {
-                    LOG.debug("vertex already build done {} ", vertex.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            vertexSerialize(serializer, vertex);
-                    loadDistributeMetrics.increaseDisVertexInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-                }
-            } else {
-                for (Edge edge : (List<Edge>) (Object) elementsElement) {
-                    LOG.debug("edge already build done {}", edge.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            edgeSerialize(serializer, edge);
-                    loadDistributeMetrics.increaseDisEdgeInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-
+            if (elementsElement != null) {
+                for (GraphElement graphElement : elementsElement) {
+                    final byte[] rowkey = serializer.getKeyBytes(graphElement);
+                    final byte[] values = serializer.getValueBytes(graphElement);
+                    ImmutableBytesWritable rowKey = new ImmutableBytesWritable();
+                    rowKey.set(rowkey);
+                    if (loadOptions.actionType.equals(Constants.DELETE_ACTION)) {
+                        KeyValue  keyValue = new KeyValue(

Review Comment:
   2 spaces in `KeyValue  keyValue`



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/direct/loader/HBaseDirectLoader.java:
##########
@@ -200,23 +201,29 @@ List<Tuple2<ImmutableBytesWritable, KeyValue>> buildAndSer(HBaseSerializer seria
                                                            struct.input().type()));
             }
 
-            boolean isVertex = builder.mapping().type().isVertex();
-            if (isVertex) {
-                for (Vertex vertex : (List<Vertex>) (Object) elementsElement) {
-                    LOG.debug("vertex already build done {} ", vertex.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            vertexSerialize(serializer, vertex);
-                    loadDistributeMetrics.increaseDisVertexInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-                }
-            } else {
-                for (Edge edge : (List<Edge>) (Object) elementsElement) {
-                    LOG.debug("edge already build done {}", edge.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            edgeSerialize(serializer, edge);
-                    loadDistributeMetrics.increaseDisEdgeInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-
+            if (elementsElement != null) {

Review Comment:
   prefer `continue if (elementsElement == null)`



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/executor/LoadOptions.java:
##########
@@ -233,6 +233,10 @@ public class LoadOptions implements Serializable {
                description = "hbaseZKParent")
     public String hbaseZKParent;
 
+    @Parameter(names = {"--action-type"},  arity = 1,

Review Comment:
   is it just for hbase? seems it's not implemented for other backends currently?



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/constant/Constants.java:
##########
@@ -81,4 +81,5 @@ public final class Constants {
     public static final String LOAD_DATA_PARSE_SUFFIX = "parse";
     public static final String LOAD_DATA_SER_SUFFIX = "ser";
     public static final String LOAD_DATA_INSERT_SUFFIX = "insert";
+    public static final String DELETE_ACTION = "delete";

Review Comment:
   prefer ACTION_DELETE, and please also define ACTION_UPDATE



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/direct/loader/HBaseDirectLoader.java:
##########
@@ -200,23 +201,29 @@ List<Tuple2<ImmutableBytesWritable, KeyValue>> buildAndSer(HBaseSerializer seria
                                                            struct.input().type()));
             }
 
-            boolean isVertex = builder.mapping().type().isVertex();
-            if (isVertex) {
-                for (Vertex vertex : (List<Vertex>) (Object) elementsElement) {
-                    LOG.debug("vertex already build done {} ", vertex.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            vertexSerialize(serializer, vertex);
-                    loadDistributeMetrics.increaseDisVertexInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-                }
-            } else {
-                for (Edge edge : (List<Edge>) (Object) elementsElement) {
-                    LOG.debug("edge already build done {}", edge.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            edgeSerialize(serializer, edge);
-                    loadDistributeMetrics.increaseDisEdgeInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-
+            if (elementsElement != null) {
+                for (GraphElement graphElement : elementsElement) {
+                    final byte[] rowkey = serializer.getKeyBytes(graphElement);
+                    final byte[] values = serializer.getValueBytes(graphElement);
+                    ImmutableBytesWritable rowKey = new ImmutableBytesWritable();
+                    rowKey.set(rowkey);
+                    if (loadOptions.actionType.equals(Constants.DELETE_ACTION)) {
+                        KeyValue  keyValue = new KeyValue(
+                                rowkey,Bytes.toBytes(Constants.HBASE_COL_FAMILY),
+                                Bytes.toBytes(Constants.EMPTY_STR),HConstants.LATEST_TIMESTAMP,
+                                KeyValue.Type.DeleteFamily);
+                        Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
+                                new Tuple2<>(rowKey,keyValue);
+                        result.add(tuple2);
+                    } else {
+                        KeyValue  keyValue = new KeyValue(rowkey,
+                                Bytes.toBytes(Constants.HBASE_COL_FAMILY),
+                                Bytes.toBytes(Constants.EMPTY_STR),
+                                values);
+                        Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
+                                new Tuple2<>(rowKey,keyValue);

Review Comment:
   ditto



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/executor/LoadOptions.java:
##########
@@ -233,6 +233,10 @@ public class LoadOptions implements Serializable {
                description = "hbaseZKParent")
     public String hbaseZKParent;

Review Comment:
   could you please also improve these names and descriptions



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/executor/LoadOptions.java:
##########
@@ -233,6 +233,10 @@ public class LoadOptions implements Serializable {
                description = "hbaseZKParent")
     public String hbaseZKParent;
 
+    @Parameter(names = {"--action-type"},  arity = 1,
+            description = "add/update or delete")

Review Comment:
   prefer "One of action types: 'update' or 'delete'"



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/direct/loader/HBaseDirectLoader.java:
##########
@@ -200,23 +201,29 @@ List<Tuple2<ImmutableBytesWritable, KeyValue>> buildAndSer(HBaseSerializer seria
                                                            struct.input().type()));
             }
 
-            boolean isVertex = builder.mapping().type().isVertex();
-            if (isVertex) {
-                for (Vertex vertex : (List<Vertex>) (Object) elementsElement) {
-                    LOG.debug("vertex already build done {} ", vertex.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            vertexSerialize(serializer, vertex);
-                    loadDistributeMetrics.increaseDisVertexInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-                }
-            } else {
-                for (Edge edge : (List<Edge>) (Object) elementsElement) {
-                    LOG.debug("edge already build done {}", edge.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            edgeSerialize(serializer, edge);
-                    loadDistributeMetrics.increaseDisEdgeInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-
+            if (elementsElement != null) {
+                for (GraphElement graphElement : elementsElement) {
+                    final byte[] rowkey = serializer.getKeyBytes(graphElement);
+                    final byte[] values = serializer.getValueBytes(graphElement);
+                    ImmutableBytesWritable rowKey = new ImmutableBytesWritable();
+                    rowKey.set(rowkey);
+                    if (loadOptions.actionType.equals(Constants.DELETE_ACTION)) {
+                        KeyValue  keyValue = new KeyValue(
+                                rowkey,Bytes.toBytes(Constants.HBASE_COL_FAMILY),
+                                Bytes.toBytes(Constants.EMPTY_STR),HConstants.LATEST_TIMESTAMP,
+                                KeyValue.Type.DeleteFamily);
+                        Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
+                                new Tuple2<>(rowKey,keyValue);

Review Comment:
   expect a space after the "rowKey,"



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/direct/loader/HBaseDirectLoader.java:
##########
@@ -200,23 +201,29 @@ List<Tuple2<ImmutableBytesWritable, KeyValue>> buildAndSer(HBaseSerializer seria
                                                            struct.input().type()));
             }
 
-            boolean isVertex = builder.mapping().type().isVertex();
-            if (isVertex) {
-                for (Vertex vertex : (List<Vertex>) (Object) elementsElement) {
-                    LOG.debug("vertex already build done {} ", vertex.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            vertexSerialize(serializer, vertex);
-                    loadDistributeMetrics.increaseDisVertexInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-                }
-            } else {
-                for (Edge edge : (List<Edge>) (Object) elementsElement) {
-                    LOG.debug("edge already build done {}", edge.toString());
-                    Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
-                            edgeSerialize(serializer, edge);
-                    loadDistributeMetrics.increaseDisEdgeInsertSuccess(builder.mapping());
-                    result.add(tuple2);
-
+            if (elementsElement != null) {
+                for (GraphElement graphElement : elementsElement) {
+                    final byte[] rowkey = serializer.getKeyBytes(graphElement);
+                    final byte[] values = serializer.getValueBytes(graphElement);
+                    ImmutableBytesWritable rowKey = new ImmutableBytesWritable();
+                    rowKey.set(rowkey);
+                    if (loadOptions.actionType.equals(Constants.DELETE_ACTION)) {
+                        KeyValue  keyValue = new KeyValue(
+                                rowkey,Bytes.toBytes(Constants.HBASE_COL_FAMILY),
+                                Bytes.toBytes(Constants.EMPTY_STR),HConstants.LATEST_TIMESTAMP,
+                                KeyValue.Type.DeleteFamily);
+                        Tuple2<ImmutableBytesWritable, KeyValue> tuple2 =
+                                new Tuple2<>(rowKey,keyValue);
+                        result.add(tuple2);
+                    } else {
+                        KeyValue  keyValue = new KeyValue(rowkey,

Review Comment:
   ditto



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] imbajin commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1233642140


##########
hugegraph-loader/assembly/static/bin/hugegraph-spark-loader.sh:
##########
@@ -25,7 +25,7 @@ get_params "$@"
 echo "engine_params: $ENGINE_PARAMS"
 echo "hugegraph_params: $HUGEGRAPH_PARAMS"
 
-ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name 'hugegraph-loader*.jar')
+ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name '*hugegraph-loader-incubating*.jar')

Review Comment:
   ```suggestion
   ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name '*hugegraph-loader*.jar')
   ```
   
   **test it** after we modify it (`incubating` word may remove in the future)



##########
hugegraph-loader/src/main/resources/META-INF/services/shaded.jakarta.ws.rs.client.ClientBuilder:
##########
@@ -0,0 +1,16 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+shaded.org.glassfish.jersey.client.JerseyClientBuilder

Review Comment:
   add a empty line for both?
   
   <img width="628" alt="image" src="https://github.com/apache/incubator-hugegraph-toolchain/assets/17706099/46d94679-ad44-4e60-a154-957a4a352fab">
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1233653691


##########
hugegraph-loader/src/main/resources/META-INF/services/shaded.jakarta.ws.rs.client.ClientBuilder:
##########
@@ -0,0 +1,16 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+shaded.org.glassfish.jersey.client.JerseyClientBuilder

Review Comment:
   ok



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1259091110


##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/executor/LoadOptions.java:
##########
@@ -233,6 +233,10 @@ public class LoadOptions implements Serializable {
                description = "hbaseZKParent")
     public String hbaseZKParent;
 
+    @Parameter(names = {"--action-type"},  arity = 1,

Review Comment:
   Yes, only hbase is implemented at this stage
   



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/constant/Constants.java:
##########
@@ -81,4 +81,5 @@ public final class Constants {
     public static final String LOAD_DATA_PARSE_SUFFIX = "parse";
     public static final String LOAD_DATA_SER_SUFFIX = "ser";
     public static final String LOAD_DATA_INSERT_SUFFIX = "insert";
+    public static final String DELETE_ACTION = "delete";

Review Comment:
   thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1233652053


##########
hugegraph-loader/assembly/static/bin/hugegraph-spark-loader.sh:
##########
@@ -25,7 +25,7 @@ get_params "$@"
 echo "engine_params: $ENGINE_PARAMS"
 echo "hugegraph_params: $HUGEGRAPH_PARAMS"
 
-ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name 'hugegraph-loader*.jar')
+ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name '*hugegraph-loader-incubating*.jar')

Review Comment:
   ok



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1261273269


##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/constant/Constants.java:
##########
@@ -81,4 +81,6 @@ public final class Constants {
     public static final String LOAD_DATA_PARSE_SUFFIX = "parse";
     public static final String LOAD_DATA_SER_SUFFIX = "ser";
     public static final String LOAD_DATA_INSERT_SUFFIX = "insert";
+    public static final String ACTION_DELETE = "delete";

Review Comment:
   there is no impact on rocksdb backend's data, because bypass server is not implemented for rocksdb backend.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] liuxiaocs7 commented on pull request #480: fix: Spark with loader has dependency conflicts

Posted by "liuxiaocs7 (via GitHub)" <gi...@apache.org>.
liuxiaocs7 commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1659739448

   Hi, @haohao0103, Wouldn't it be better to separate the two things in different PRs?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] codecov[bot] commented on pull request #480: fix Spark loader has dependency conflicts

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1590501719

   ## [Codecov](https://app.codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/480?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   > Merging [#480](https://app.codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/480?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (bc2c8e9) into [master](https://app.codecov.io/gh/apache/incubator-hugegraph-toolchain/commit/f0cbdbcf5d2702f6784830a4f964c08660b8cda4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (f0cbdbc) will **decrease** coverage by `0.05%`.
   > The diff coverage is `n/a`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master     #480      +/-   ##
   ============================================
   - Coverage     62.48%   62.44%   -0.05%     
   + Complexity     1868      895     -973     
   ============================================
     Files           260       91     -169     
     Lines          9432     4402    -5030     
     Branches        875      518     -357     
   ============================================
   - Hits           5894     2749    -3145     
   + Misses         3154     1448    -1706     
   + Partials        384      205     -179     
   ```
   
   
   [see 169 files with indirect coverage changes](https://app.codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/480/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] imbajin commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1281397899


##########
hugegraph-loader/assembly/static/bin/hugegraph-spark-loader.sh:
##########
@@ -25,7 +25,7 @@ get_params "$@"
 echo "engine_params: $ENGINE_PARAMS"
 echo "hugegraph_params: $HUGEGRAPH_PARAMS"
 
-ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name 'hugegraph-loader*.jar')
+ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name '*hugegraph-loader-incubating*.jar')

Review Comment:
   > ok
   
   seems also revert the `incubating` that we should remove? (Or we can't remove it after testing?)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] imbajin commented on pull request #480: fix: Spark with loader has dependency conflicts

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1596137649

   @haohao0103 please update the branch with the latest code & fix the conflicts & then test the bug again


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] javeme commented on a diff in pull request #480: fix Spark loader has dependency conflicts

Posted by "javeme (via GitHub)" <gi...@apache.org>.
javeme commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1229810027


##########
hugegraph-loader/pom.xml:
##########
@@ -41,12 +41,18 @@
         <final.name>apache-${release.name}-incubating-${project.version}</final.name>
         <jackson.version>2.12.3</jackson.version>
         <hadoop.version>3.3.1</hadoop.version>
+        <gson.version>2.8.5</gson.version>
         <assembly.dir>${project.basedir}/assembly</assembly.dir>
         <assembly.descriptor.dir>${assembly.dir}/descriptor</assembly.descriptor.dir>
         <assembly.static.dir>${assembly.dir}/static</assembly.static.dir>
     </properties>
 
     <dependencies>
+        <dependency>

Review Comment:
   prefer to move to line 99



##########
hugegraph-loader/pom.xml:
##########
@@ -612,6 +622,55 @@
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-compiler-plugin</artifactId>
             </plugin>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-shade-plugin</artifactId>
+                <version>3.4.1</version>
+                <executions>
+                    <execution>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>shade</goal>
+                        </goals>
+                        <configuration>
+                            <relocations>
+                                <relocation>
+                                    <pattern>org.glassfish.jersey</pattern>
+                                    <shadedPattern>shaded.org.glassfish.jersey</shadedPattern>
+                                </relocation>
+                                <relocation>
+                                    <pattern>jakarta.ws.rs</pattern>

Review Comment:
   please note the [checkStatus()](https://github.com/apache/incubator-hugegraph-commons/blob/master/hugegraph-common/src/main/java/org/apache/hugegraph/rest/AbstractRestClient.java#L193) is an abstract method, I'm not sure the shaded class with `shaded.jakarta.ws.rs` package is still named `jakarta.ws.rs.core.Response`.



##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java:
##########
@@ -293,8 +293,8 @@ private void parse(Row row, Map.Entry<ElementBuilder, List<GraphElement>> builde
             case HDFS:
                 FileSource fileSource = struct.input().asFileSource();
                 String delimiter = fileSource.delimiter();
-                elements = builder.build(fileSource.header(),
-                                         row.mkString(delimiter).split(delimiter));
+
+                elements = builder.build(row);

Review Comment:
   not sure why changed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] imbajin commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1230647910


##########
hugegraph-loader/assembly/static/bin/hugegraph-spark-loader.sh:
##########
@@ -25,7 +25,7 @@ get_params "$@"
 echo "engine_params: $ENGINE_PARAMS"
 echo "hugegraph_params: $HUGEGRAPH_PARAMS"
 
-ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name 'hugegraph-loader*.jar')
+ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name 'apache-hugegraph-loader-incubating*.jar')

Review Comment:
   ```suggestion
   ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name '*hugegraph-loader-incubating*.jar')
   ```
   
   try this way if test **OK**, just a suggestion, not required (to avoid filename changed)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on pull request #480: fix Spark loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1590981581

   @imbajin @simon824 @JackyYangPassion Could you give me some advice? tanks.The error unittest I can pass in the local .
   <img width="1067" alt="image" src="https://github.com/apache/incubator-hugegraph-toolchain/assets/30570711/e4f073b9-813f-42a9-b1a6-f19f27a7c763">
   ![image](https://github.com/apache/incubator-hugegraph-toolchain/assets/30570711/e779b0a5-f864-4f14-9fcf-6dead7c2b133)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] imbajin commented on pull request #480: fix Spark loader has dependency conflicts

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#issuecomment-1591657532

   > @imbajin @simon824 @JackyYangPassion Could you give me some advice? thanks.The error unittest I can pass in the local . <img alt="image" width="1067" src="https://user-images.githubusercontent.com/30570711/245770836-e4f073b9-813f-42a9-b1a6-f19f27a7c763.png"> ![image](https://user-images.githubusercontent.com/30570711/245771179-e779b0a5-f864-4f14-9fcf-6dead7c2b133.png)
   
   could rebase/merge #478 first (will also fix the `dependency-review` problem)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] simon824 merged pull request #480: fix: Spark with loader has dependency conflicts

Posted by "simon824 (via GitHub)" <gi...@apache.org>.
simon824 merged PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1281462794


##########
hugegraph-loader/assembly/static/bin/hugegraph-spark-loader.sh:
##########
@@ -25,7 +25,7 @@ get_params "$@"
 echo "engine_params: $ENGINE_PARAMS"
 echo "hugegraph_params: $HUGEGRAPH_PARAMS"
 
-ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name 'hugegraph-loader*.jar')
+ASSEMBLY_JAR_NAME=$(find "${LIB_DIR}" -name '*hugegraph-loader-incubating*.jar')

Review Comment:
   I will remove it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


[GitHub] [incubator-hugegraph-toolchain] haohao0103 commented on a diff in pull request #480: fix: Spark with loader has dependency conflicts

Posted by "haohao0103 (via GitHub)" <gi...@apache.org>.
haohao0103 commented on code in PR #480:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/480#discussion_r1284239374


##########
hugegraph-loader/src/main/resources/META-INF/services/shaded.jakarta.ws.rs.client.ClientBuilder:
##########
@@ -0,0 +1,16 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+shaded.org.glassfish.jersey.client.JerseyClientBuilder

Review Comment:
   hello, I understand that it is because  SPI  added these two classes in jersey-client jar and other jars. When we do shade, we are actually changing the path information of these two classes, so we need to add SPI to these two shade classes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org