[jira] [Commented] (S2GRAPH-226) Provide example spark jobs to explain how to utilize WAL log.

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/S2GRAPH-226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16711400#comment-16711400 ] 

ASF GitHub Bot commented on S2GRAPH-226:
----------------------------------------

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-s2graph/pull/182


> Provide example spark jobs to explain how to utilize WAL log.
> -------------------------------------------------------------
>
>                 Key: S2GRAPH-226
>                 URL: https://issues.apache.org/jira/browse/S2GRAPH-226
>             Project: S2Graph
>          Issue Type: New Feature
>          Components: s2core, s2jobs
>            Reporter: DOYUNG YOON
>            Assignee: DOYUNG YOON
>            Priority: Minor
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> Even though s2graph publish all incoming vertex/edge into Kafka, there is no example showing how to use this WAL log.
> I suggest adding a simple example showing how to process WAL and let me explain what use cases this example can benefit.
> At kakao, s2graph have been used as the fact storage, which store all user's activities such as click content, buy a product, search query.
> {noformat}
> [{
> 	"timestamp": 1,
> 	"elem": "e",
> 	"from": "steamshon",
> 	"to": "s2graph",
> 	"label": "search_query",
> 	"props": {}
> }, {
> 	"timestamp": 10,
> 	"elem": "e",
> 	"from": "steamshon",
> 	"to": "github.com/apache/incubator-s2graph",
> 	"label": "content_click",
> 	"props": {}
> }, {
> 	"timestamp": 12,
> 	"elem": "v",
> 	"id": "steamshon",
> 	"serviceName": "s2graph",
> 	"columnName": "user",
> 	"props": {
> 		"gender": "M"
> 	}
> }]
> {noformat}
> Each activity, label in s2graph words, consisting of their own graph, but when they are all connected together, then it gives much more information. 
> Above edges can be aggregated as Vertex.
> It is up to users how to connect each graph, but in our case, we used `user` to merge multiple graphs. for example, we made each activity such as click content, buy a product, search query all use the same `userId` for the same `user`. 
> Below is simple example data.
> {noformat}
> {
> 	"timestamp": 10,
> 	"elem": "v",
> 	"id": "steamshon",
> 	"serviceName": "s2graph",
> 	"columnName": "user",
> 	"props": {
> 		"gender": "M",
> 		"edges": [{
> 			"timestamp": 1,
> 			"to": "s2graph",
> 			"label": "search_query",
> 			"props": {}
> 		}, {
> 			"timestamp": 10,
> 			"to": "github.com/apache/incubator-s2graph",
> 			"label": "content_click",
> 			"props": {}
> 		}]
> 	}
> }
> {noformat}
> This connected graph can be used not only for OLTP but also OLAP.
> I believe s2graph WAL log is good way to integrate OLTP and OLAP, and adding this example can help for user to understand how to leverage it.
> [desing doc(work in progress)|https://docs.google.com/document/d/12vsJFWTKoDrYxD-NhOoCaJdvFg8it2wYqIABA0DXYeA/edit?usp=sharing] 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)