[jira] [Commented] (TINKERPOP3-320) BulkDumpingMapReduce (proposal)

["Daniel Kuppitz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TINKERPOP3-320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14975094#comment-14975094 ] 

Daniel Kuppitz commented on TINKERPOP3-320:
-------------------------------------------

I think I have it (almost) working. The only thing I'm not sure about is {{.output('dir/')}}. How do I know whether {{dir}} is a local directory or an HDFS directory?

> BulkDumpingMapReduce (proposal)
> -------------------------------
>
>                 Key: TINKERPOP3-320
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP3-320
>             Project: TinkerPop 3
>          Issue Type: Improvement
>          Components: process, structure
>    Affects Versions: 3.0.2-incubating
>            Reporter: Marko A. Rodriguez
>            Assignee: Daniel Kuppitz
>
> This is another trivial "bulker" that is vendor agnostic. For any {{Graph}} that provides a {{GraphComputer}}, {{BulkDumpingMapReduce}} is executed as such:
> {code}
> g.compute().mapReduce(BulkDumpingMapReduce.build().output('dir/').writer(GraphSONWriter.class).create()).submit()
> {code}
> The implementation is:
> {code}
> public BulkDumpingMapReduce implements MapReduce {
>    public void map(vertex, emitter) {
>      StringBufferThingy vertexString = new StringBufferThingy()  
>      graphSONWriter.write(vertex,stringBufferStreamThingy);
>      emitter.emit(vertexString.toString());
>    }
> }
> {code}
> NOTES:
> * MapReduce generates a key/value pair result set. Thus, {{MapReduce.generateResult()}} would need to clip off the key and write the value to the output location.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)