You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Lukasz Gajowy (Jira)" <ji...@apache.org> on 2019/10/28 18:10:00 UTC

[jira] [Commented] (BEAM-6303) Add .parquet extension to files in ParquetIO

    [ https://issues.apache.org/jira/browse/BEAM-6303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961315#comment-16961315 ] 

Lukasz Gajowy commented on BEAM-6303:
-------------------------------------

There is an easy way to provide the extension right now:
{code:java}
FileIO.<GenericRecord>write()
    .via(ParquetIO.sink(SCHEMA))
    .to(filenamePrefix)
    .withSuffix(".parquet")){code}
Other file ios (TfrecordIO, AvroIO, TextIO) do not set the default extension as well and their javadoc comments suggest using withSuffix() method as well. 

If we don't want to decorate the sink with the suffix by wrapping it in PTransform (imho there's no need to do that for the suffix only), the best solution is to update the javadoc comment in ParquetIO.

> Add .parquet extension to files in ParquetIO
> --------------------------------------------
>
>                 Key: BEAM-6303
>                 URL: https://issues.apache.org/jira/browse/BEAM-6303
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-parquet
>            Reporter: Lukasz Gajowy
>            Assignee: Lukasz Gajowy
>            Priority: Major
>
> There should be .parquet extension added by default when writing files with ParquetIO



--
This message was sent by Atlassian Jira
(v8.3.4#803005)