You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Masayuki Takahashi (JIRA)" <ji...@apache.org> on 2018/05/12 12:45:00 UTC
[jira] [Updated] (PARQUET-1297) [Java] SchemaConverter should not
convert from Timestamp(TimeUnit.SECOND) and Timestamp(TimeUnit.NANOSECOND)
of Arrow
[ https://issues.apache.org/jira/browse/PARQUET-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Masayuki Takahashi updated PARQUET-1297:
----------------------------------------
Description:
Arrow's 'Timestamp' definition is below:
{code:java}
{ "name" : "time", "unit" : "SECOND|MILLISECOND|MICROSECOND|NANOSECOND",}
{code}
[http://arrow.apache.org/docs/metadata.html]
But Parquet only supports 'TIMESTAMP_MILLIS' and 'TIMESTAMP_MICROS'.
[https://github.com/Apache/parquet-format/blob/master/LogicalTypes.md]
Therefore SchemaConverter should not convert from Timestamp(TimeUnit.SECOND) and Timestamp(TimeUnit.NANOSECOND) of Arrow to Parquet.
Related:
https://issues.apache.org/jira/browse/PARQUET-1285
was:
Arrow's 'Time' definition is below:
{code:java}
{ "name" : "time", "unit" : "SECOND|MILLISECOND|MICROSECOND|NANOSECOND", "bitWidth": /* integer: 32 or 64 */ }{code}
[http://arrow.apache.org/docs/metadata.html]
But Parquet only supports 'TIME_MILLIS' and 'TIME_MICROS'.
[https://github.com/Apache/parquet-format/blob/master/LogicalTypes.md]
Therefore SchemaConverter should not convert from TimeUnit.SECOND AND TimeUnit.NANOSECOND of Arrow to Parquet.
> [Java] SchemaConverter should not convert from Timestamp(TimeUnit.SECOND) and Timestamp(TimeUnit.NANOSECOND) of Arrow
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: PARQUET-1297
> URL: https://issues.apache.org/jira/browse/PARQUET-1297
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Reporter: Masayuki Takahashi
> Assignee: Masayuki Takahashi
> Priority: Minor
> Fix For: 1.10.0
>
>
> Arrow's 'Timestamp' definition is below:
> {code:java}
> { "name" : "time", "unit" : "SECOND|MILLISECOND|MICROSECOND|NANOSECOND",}
> {code}
> [http://arrow.apache.org/docs/metadata.html]
> But Parquet only supports 'TIMESTAMP_MILLIS' and 'TIMESTAMP_MICROS'.
> [https://github.com/Apache/parquet-format/blob/master/LogicalTypes.md]
> Therefore SchemaConverter should not convert from Timestamp(TimeUnit.SECOND) and Timestamp(TimeUnit.NANOSECOND) of Arrow to Parquet.
> Related:
> https://issues.apache.org/jira/browse/PARQUET-1285
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)