You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hugegraph.apache.org by ji...@apache.org on 2022/08/13 08:33:54 UTC

[incubator-hugegraph-doc] branch master updated (9ddce273 -> cf2f878c)

This is an automated email from the ASF dual-hosted git repository.

jin pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hugegraph-doc.git


 discard 9ddce273 1
 discard 9faf633e spark-loader doc
     new cf2f878c doc: usage about spark-loader module

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (9ddce273)
            \
             N -- N -- N   refs/heads/master (cf2f878c)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:


[incubator-hugegraph-doc] 01/01: doc: usage about spark-loader module

Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

jin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hugegraph-doc.git

commit cf2f878c886d024e5bafcdff2871e736b6a2d105
Author: Simon <mi...@apache.org>
AuthorDate: Mon Aug 1 15:26:22 2022 +0800

    doc: usage about spark-loader module
---
 content/cn/docs/quickstart/hugegraph-loader.md | 15 +++++++++++++++
 content/en/docs/quickstart/hugegraph-loader.md | 17 +++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/content/cn/docs/quickstart/hugegraph-loader.md b/content/cn/docs/quickstart/hugegraph-loader.md
index a7aa60e0..49ba937a 100644
--- a/content/cn/docs/quickstart/hugegraph-loader.md
+++ b/content/cn/docs/quickstart/hugegraph-loader.md
@@ -947,3 +947,18 @@ count metrics
      edge insert failure           : 0
 ```
 
+#### 4.5 使用 spark-loader 导入
+> Spark 版本:Spark 3+, 其他版本未测试。
+> HugeGraph Toolchain 版本: toolchain-1.0.0
+> 
+`spark-loader` 的参数分为两部分,注意:因二者参数名缩写存在重合部分,请使用参数全称。两种参数之间无需保证先后顺序。
+- hugegraph 参数(参考:[hugegraph-loader 参数说明](https://hugegraph.apache.org/cn/docs/quickstart/hugegraph-loader/#341-%E5%8F%82%E6%95%B0%E8%AF%B4%E6%98%8E) )
+- Spark 任务提交参数 (参考:[Submitting Applications](https://spark.apache.org/docs/3.3.0/submitting-applications.html#content))
+
+示例:
+```bash
+sh bin/hugegraph-spark-loader.sh --master yarn \
+--deploy-mode cluster --name spark-hugegraph-loader --file ./hugegraph.json \
+--username admin --token admin --host xx.xx.xx.xx --port 8093 \
+--graph graph-test --num-executors 6 --executor-cores 16 --executor-memory 15g
+```
diff --git a/content/en/docs/quickstart/hugegraph-loader.md b/content/en/docs/quickstart/hugegraph-loader.md
index 32e666bc..b737aac5 100644
--- a/content/en/docs/quickstart/hugegraph-loader.md
+++ b/content/en/docs/quickstart/hugegraph-loader.md
@@ -944,3 +944,20 @@ count metrics
      edge insert failure           : 0
 ```
 
+#### 4.5 Import data by spark-loader
+> Spark version: Spark 3+, other versions has not been tested.  
+> HugeGraph Toolchain version: toolchain-1.0.0
+> 
+The parameters of `spark-loader` are divided into two parts. Note: Because the abbreviations of 
+these two parameter names have overlapping parts, please use the full name of the parameter. 
+And there is no need to guarantee the order between the two parameters.
+- hugegraph parameters (Reference: [hugegraph-loader parameter description](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#341-parameter-description) )
+- Spark task submission parameters (Reference: [Submitting Applications](https://spark.apache.org/docs/3.3.0/submitting-applications.html#content))
+
+Example:
+```bash
+sh bin/hugegraph-spark-loader.sh --master yarn \
+--deploy-mode cluster --name spark-hugegraph-loader --file ./hugegraph.json \
+--username admin --token admin --host xx.xx.xx.xx --port 8093 \
+--graph graph-test --num-executors 6 --executor-cores 16 --executor-memory 15g
+````