You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@s2graph.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/06/15 01:54:00 UTC
[jira] [Commented] (S2GRAPH-221) Unify configurations for bulk and
mutate in S2GraphSink.
[ https://issues.apache.org/jira/browse/S2GRAPH-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513234#comment-16513234 ]
ASF GitHub Bot commented on S2GRAPH-221:
----------------------------------------
GitHub user SteamShon opened a pull request:
https://github.com/apache/incubator-s2graph/pull/173
[S2GRAPH-221]: Unify configurations for bulk and mutate in S2GraphSink.
- cleanup configurations for S2Graph, S2GraphSink, S2GraphSource.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SteamShon/incubator-s2graph S2GRAPH-221
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-s2graph/pull/173.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #173
----
commit e42f7b22f26c7d725f9a33ff6badd7942768e478
Author: DO YUNG YOON <st...@...>
Date: 2018-06-14T10:20:51Z
initial commit.
commit f90a7f54401f67130360ca711a3378a726aeac4b
Author: DO YUNG YOON <st...@...>
Date: 2018-06-14T20:04:35Z
test cases passed.
commit da668ace749adf243d43c95413b460bba218731d
Author: DO YUNG YOON <st...@...>
Date: 2018-06-15T01:09:53Z
commit missing changes on s2core.
----
> Unify configurations for bulk and mutate in S2GraphSink.
> --------------------------------------------------------
>
> Key: S2GRAPH-221
> URL: https://issues.apache.org/jira/browse/S2GRAPH-221
> Project: S2Graph
> Issue Type: Improvement
> Components: s2jobs
> Reporter: DOYUNG YOON
> Assignee: DOYUNG YOON
> Priority: Minor
>
> Currently, S2GraphSink in s2jobs provide two way to sink data from spark into S2Graph.
> 1. mutate: open S2Graph per each executor, then call mutateElements method.
> 2. bulk: run spark job to build HFile and run loadIncrementalHFiles.
> It is hard to followup options for these two different method, since mutate options are defined in org.apache.s2graph.spark.sql.streaming.S2SinkConfigs, and bulkload options are defined in org.apache.s2graph.s2jobs.load.GraphFileOptions.
> I suggest placing all configurations under in one place so it would be easy to maintain them.
> Also, many options for bulk can be removed.
> One example is dbUrl options, which is same as "db.default.url" and zkQuorum, which is same as "hbase.zookeeper.quorum".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)