You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@s2graph.apache.org by st...@apache.org on 2018/04/02 03:32:58 UTC
[3/3] incubator-s2graph git commit: [S2GRAPH-183]: Provide batch job
to dump data stored in HBase into file.
[S2GRAPH-183]: Provide batch job to dump data stored in HBase into file.
JIRA:
[S2GRAPH-183] https://issues.apache.org/jira/browse/S2GRAPH-183
Pull Request:
Closes #137
Author
DO YUNG YOON <st...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/commit/3332f6bc
Tree: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/tree/3332f6bc
Diff: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/diff/3332f6bc
Branch: refs/heads/master
Commit: 3332f6bc13cc753ccb2e6d69ecb954758985f0da
Parents: 5b22148 dde0268
Author: DO YUNG YOON <st...@apache.org>
Authored: Mon Apr 2 12:31:16 2018 +0900
Committer: DO YUNG YOON <st...@apache.org>
Committed: Mon Apr 2 12:32:09 2018 +0900
----------------------------------------------------------------------
CHANGES | 1 +
.../scala/org/apache/s2graph/core/S2Graph.scala | 2 +
.../apache/s2graph/core/storage/SKeyValue.scala | 5 +-
.../hbase/AsynchbaseStorageManagement.scala | 2 +-
.../org/apache/s2graph/s2jobs/DegreeKey.scala | 52 ++++
.../apache/s2graph/s2jobs/S2GraphHelper.scala | 50 ++++
.../s2jobs/loader/GraphFileGenerator.scala | 8 +-
.../s2graph/s2jobs/loader/HFileGenerator.scala | 126 +--------
.../s2jobs/loader/HFileMRGenerator.scala | 34 +--
.../loader/LocalBulkLoaderTransformer.scala | 61 ++++
.../s2jobs/loader/RawFileGenerator.scala | 28 +-
.../loader/SparkBulkLoaderTransformer.scala | 76 +++++
.../s2jobs/serde/GraphElementReadable.scala | 26 ++
.../s2jobs/serde/GraphElementWritable.scala | 26 ++
.../s2graph/s2jobs/serde/Transformer.scala | 50 ++++
.../serde/reader/TsvBulkFormatReader.scala | 29 ++
.../s2jobs/serde/writer/KeyValueWriter.scala | 33 +++
.../apache/s2graph/s2jobs/BaseSparkTest.scala | 145 ++++++++++
.../s2jobs/dump/GraphFileDumperTest.scala | 97 +++++++
.../s2jobs/loader/GraphFileGeneratorTest.scala | 277 +++++++------------
20 files changed, 783 insertions(+), 345 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-s2graph/blob/3332f6bc/CHANGES
----------------------------------------------------------------------
diff --cc CHANGES
index cc5b308,c44c8c0..39360ba
--- a/CHANGES
+++ b/CHANGES
@@@ -72,7 -61,6 +72,8 @@@ Release Notes - S2Graph - Version 0.2.
* [S2GRAPH-175] - Provide Elastic Search Index Provider.
* [S2GRAPH-172] - Suggest to implement GraphQL as standard web interface for S2Graph.
* [S2GRAPH-177] - Add support for createServiceColumn/addVertex APIs on GraphQL.
+ * [S2GRAPH-185] - Support Spark Structured Streaming to work with data in streaming and batch.
++ * [S2GRAPH-183] - Provide batch job to dump data stored in HBase into file.
** Task
* [S2GRAPH-162] - Update year in the NOTICE file.
http://git-wip-us.apache.org/repos/asf/incubator-s2graph/blob/3332f6bc/s2jobs/src/main/scala/org/apache/s2graph/s2jobs/loader/GraphFileGenerator.scala
----------------------------------------------------------------------