[jira] [Commented] (TINKERPOP-1298) Save OLAP results to file

["Daniel Kuppitz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TINKERPOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15298817#comment-15298817 ] 

Daniel Kuppitz commented on TINKERPOP-1298:
-------------------------------------------

Because I'm sure that I will forget anything that I just learned about global traversers, here's how {{BulkExportVertexProgram}} should approximately look like in 3.3:

https://gist.github.com/dkuppitz/4ee35f89353978ac6d7428cbc71305d2

For now {{BulkExportVertexProgram}} supports exports for almost any traversal that doesn't end with a reducing barrier step. In order to support the latter, we depend on a solution for these 2 tickets:

* https://issues.apache.org/jira/browse/TINKERPOP-1309
* https://issues.apache.org/jira/browse/TINKERPOP-1306

It relies on {{ScriptOutputFormat}} to write a tabular / comma-separated / clear-text output. A sample configuration and Groovy script for the {{ScriptOutputFormat}} will be provided as part of the TinkerPop distribution. I will add an integration test and a fully working example in the docs before I create a PR.

> Save OLAP results to file
> -------------------------
>
>                 Key: TINKERPOP-1298
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-1298
>             Project: TinkerPop
>          Issue Type: Improvement
>          Components: io, process
>            Reporter: Daniel Kuppitz
>            Assignee: Daniel Kuppitz
>
> Provide a way to save (tabular) results to text files, just like Spark's {{saveAsTextFile}}.
> I'm not sure about the best way to do it. 3 options come to my mind:
> # a new step.
> # a {{VertexProgram}}
> # a configuration option
> Things to consider / open questions:
> * Is it sufficient to simply {{toString()}} all values or should we allow formatters / format stings?
> * [~jlewandowski] pointed out that it would be nice to have support for the [parquet file format|https://parquet.apache.org/]. I guess now we're already talking about support for different {{FileOutputFormats}} and not just formatters.
> * Is that only relevant for OLAP?
> * Can we support arbitrary file systems?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)