You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hugegraph.apache.org by ji...@apache.org on 2022/08/13 08:27:38 UTC

[incubator-hugegraph-doc] 01/02: spark-loader doc

This is an automated email from the ASF dual-hosted git repository.

jin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hugegraph-doc.git

commit 9faf633e1f6afe178e706f27f25cd604643eb659
Author: Simon <mi...@apache.org>
AuthorDate: Mon Aug 1 15:26:22 2022 +0800

    spark-loader doc
---
 content/cn/docs/quickstart/hugegraph-loader.md | 14 ++++++++++++++
 content/en/docs/quickstart/hugegraph-loader.md | 16 ++++++++++++++++
 2 files changed, 30 insertions(+)

diff --git a/content/cn/docs/quickstart/hugegraph-loader.md b/content/cn/docs/quickstart/hugegraph-loader.md
index a7aa60e0..7c29d471 100644
--- a/content/cn/docs/quickstart/hugegraph-loader.md
+++ b/content/cn/docs/quickstart/hugegraph-loader.md
@@ -947,3 +947,17 @@ count metrics
      edge insert failure           : 0
 ```
 
+#### 4.5 使用 spark-loader 导入
+> Spark 版本:Spark 3+, 其他版本未测试。
+> 
+`spark-loader` 的参数分为两部分,注意:因二者参数名缩写存在重合部分,请使用参数全称。两种参数之间无需保证先后顺序。
+- hugegraph 参数(参考:[hugegraph-loader 参数说明](https://hugegraph.apache.org/cn/docs/quickstart/hugegraph-loader/#341-%E5%8F%82%E6%95%B0%E8%AF%B4%E6%98%8E) )
+- Spark 任务提交参数 (参考:[Submitting Applications](https://spark.apache.org/docs/3.3.0/submitting-applications.html#content))
+
+示例:
+```bash
+sh bin/hugegraph-spark-loader.sh --master yarn \
+--deploy-mode cluster --name spark-hugegraph-loader --file ./hugegraph.json \
+--username admin --token admin --host xx.xx.xx.xx --port 8093 \
+--graph graph-test --num-executors 6 --executor-cores 16 --executor-memory 15g
+```
diff --git a/content/en/docs/quickstart/hugegraph-loader.md b/content/en/docs/quickstart/hugegraph-loader.md
index 32e666bc..90e2beda 100644
--- a/content/en/docs/quickstart/hugegraph-loader.md
+++ b/content/en/docs/quickstart/hugegraph-loader.md
@@ -944,3 +944,19 @@ count metrics
      edge insert failure           : 0
 ```
 
+#### 4.5 Import data by spark-loader
+> Spark version: Spark 3+, other versions has not been tested.  
+> 
+The parameters of `spark-loader` are divided into two parts. Note: Because the abbreviations of 
+these two parameter names have overlapping parts, please use the full name of the parameter. 
+And there is no need to guarantee the order between the two parameters.
+- hugegraph parameters (Reference: [hugegraph-loader parameter description](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#341-parameter-description) )
+- Spark task submission parameters (Reference: [Submitting Applications](https://spark.apache.org/docs/3.3.0/submitting-applications.html#content))
+
+Example:
+```bash
+sh bin/hugegraph-spark-loader.sh --master yarn \
+--deploy-mode cluster --name spark-hugegraph-loader --file ./hugegraph.json \
+--username admin --token admin --host xx.xx.xx.xx --port 8093 \
+--graph graph-test --num-executors 6 --executor-cores 16 --executor-memory 15g
+````