You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by GitBox <gi...@apache.org> on 2019/07/16 15:25:59 UTC
[GitHub] [gora] LahiruJayasekara opened a new pull request #175: GORA-546
Hazelcast Jet execution engine support
LahiruJayasekara opened a new pull request #175: GORA-546 Hazelcast Jet execution engine support
URL: https://github.com/apache/gora/pull/175
Don't merge
Please review
Your feed back is mostly welcome
In this PR I have implemented the new jet source and sink connectors for Gora.
I have added some test cases to check the functionality of those connectors.
Following is an example query for word counting using the new connectors.
```
JetEngine<Long, Pageview, Long, ResultPageView> jetEngine = new JetEngine<>();
BatchSource<JetInputOutputFormat<Long, Pageview>> fileSource = jetEngine.createDataSource(dataStore, query);
Pipeline p = Pipeline.create();
p.drawFrom(fileSource)
.filter(item -> item.getValue().getIp().toString().equals("88.240.129.183"))
.map(e -> {
ResultPageView resultPageView = new ResultPageView();
resultPageView.setIp(e.getValue().getIp());
resultPageView.setTimestamp(e.getValue().getTimestamp());
resultPageView.setUrl(e.getValue().getUrl());
return new JetInputOutputFormat<Long, ResultPageView>(e.getValue().getTimestamp(), resultPageView);
})
.drainTo(jetEngine.createDataSink(dataStoreOut));
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services