[jira] [Created] (ARROW-7678) [C++][Parquet] setting TZ= in environment on Linux causes broken parquet

["Joshua Pedrick (Jira)" <ji...@apache.org>]

Joshua Pedrick created ARROW-7678:
-------------------------------------

             Summary: [C++][Parquet] setting TZ= in environment on Linux causes broken parquet
                 Key: ARROW-7678
                 URL: https://issues.apache.org/jira/browse/ARROW-7678
             Project: Apache Arrow
          Issue Type: Bug
          Components: C++
    Affects Versions: 0.15.1
         Environment: Linux, Ubuntu 18.04, arrow/parquet 0.15.1 from instructions https://arrow.apache.org/install/
            Reporter: Joshua Pedrick


When I set TZ=CST-8, or other timezone on Linux time columns are corrupted in my resulting parquet file.

 

Below are the calls I use to define my schema:

 
{code:java}
PrimitiveNode::Make( columnName, Repetition::REQUIRED,
 LogicalType::Timestamp( true, LogicalType::TimeUnit::MICROS, false, false ),
 ::parquet::Type::INT64 ) );
PrimitiveNode::Make( columnName,
 repetition,
 LogicalType::Time( true, LogicalType::TimeUnit::MICROS ),
 ::parquet::Type::INT64 ) );
{code}
I use an Int64Writer for both types. When reading, in this case using pandas with pyarrow, but also in C++, I get the following exception:
{code:java}
 File "pyarrow/_parquet.pyx", line 1136, in pyarrow._parquet.ParquetReader.read_all
 File "pyarrow/error.pxi", line 80, in pyarrow.lib.check_status
pyarrow.lib.ArrowIOError: Couldn't deserialize thrift: TProtocolException: Invalid data
Deserializing page header failed.{code}
Seems as if the column header must be defining a timestamp+timezone even though I manually set is_adjusted_to_utc.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)