You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Vitalii Diravka (JIRA)" <ji...@apache.org> on 2018/07/12 15:56:00 UTC
[jira] [Commented] (DRILL-4742) Using convert_from timestamp_impala
gives a random error
[ https://issues.apache.org/jira/browse/DRILL-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541852#comment-16541852 ]
Vitalii Diravka commented on DRILL-4742:
----------------------------------------
The issue is not reproduced anymore.
{code}
0: jdbc:drill:zk=local> select * from dfs.`/home/vitalii/Downloads/temp.parquet` limit 2;
+-----------+------------------+------+---------------+----------------+------------+-------------------+--------------+
| voter_id | name | age | registration | contributions | voterzone | create_timestamp | create_date |
+-----------+------------------+------+---------------+----------------+------------+-------------------+--------------+
| 1 | wendy van buren | 22 | republican | 168.4 | 14673 | [B@727782da | 2016-07-05 |
| 2 | sarah young | 33 | democrat | 757.74 | 13104 | [B@36430024 | 2016-04-28 |
+-----------+------------------+------+---------------+----------------+------------+-------------------+--------------+
2 rows selected (0.297 seconds)
0: jdbc:drill:zk=local> select CONVERT_FROM(create_timestamp, 'TIMESTAMP_IMPALA') from dfs.`/home/vitalii/Downloads/temp.parquet` limit 2;
+------------------------+
| EXPR$0 |
+------------------------+
| 2016-10-24 06:03:58.0 |
| 2016-12-08 22:58:14.0 |
+------------------------+
2 rows selected (0.142 seconds)
0: jdbc:drill:zk=local> set `store.parquet.reader.int96_as_timestamp` = true;
+-------+---------------------------------------------------+
| ok | summary |
+-------+---------------------------------------------------+
| true | store.parquet.reader.int96_as_timestamp updated. |
+-------+---------------------------------------------------+
1 row selected (0.06 seconds)
0: jdbc:drill:zk=local> select * from dfs.`/home/vitalii/Downloads/temp.parquet` limit 2;
+-----------+------------------+------+---------------+----------------+------------+------------------------+--------------+
| voter_id | name | age | registration | contributions | voterzone | create_timestamp | create_date |
+-----------+------------------+------+---------------+----------------+------------+------------------------+--------------+
| 1 | wendy van buren | 22 | republican | 168.4 | 14673 | 2016-10-24 06:03:58.0 | 2016-07-05 |
| 2 | sarah young | 33 | democrat | 757.74 | 13104 | 2016-12-08 22:58:14.0 | 2016-04-28 |
+-----------+------------------+------+---------------+----------------+------------+------------------------+--------------+
2 rows selected (0.228 seconds)
{code}
See more in DRILL-4337
> Using convert_from timestamp_impala gives a random error
> --------------------------------------------------------
>
> Key: DRILL-4742
> URL: https://issues.apache.org/jira/browse/DRILL-4742
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.6.0, 1.7.0
> Reporter: Rahul Challapalli
> Priority: Critical
> Attachments: error.txt, temp.parquet
>
>
> Drill Commit # fbdd20e54351879200184b478c2a32f238bf2176
> The following query randomly generates the below error.
> {code}
> select convert_from(create_timestamp, 'TIMESTAMP_IMPALA') from dfs.`/drill/testdata/temp.parquet`;
> Error: SYSTEM ERROR: ArrayIndexOutOfBoundsException: 0
> Fragment 0:0
> [Error Id: 9fe53a95-c4ae-424d-8c6d-489abab2d2ca on qa-node190.qa.lab:31010] (state=,code=0)
> {code}
> The underlying parquet file is generated using hive. Below is the metadata information
> {code}
> /root/parquet-tools-1.5.1-SNAPSHOT/parquet-meta temp.parquet
> creator: parquet-mr version 1.6.0
> file schema: hive_schema
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> voter_id: OPTIONAL INT32 R:0 D:1
> name: OPTIONAL BINARY O:UTF8 R:0 D:1
> age: OPTIONAL INT32 R:0 D:1
> registration: OPTIONAL BINARY O:UTF8 R:0 D:1
> contributions: OPTIONAL FLOAT R:0 D:1
> voterzone: OPTIONAL INT32 R:0 D:1
> create_timestamp: OPTIONAL INT96 R:0 D:1
> create_date: OPTIONAL INT32 O:DATE R:0 D:1
> row group 1: RC:200 TS:9902
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> voter_id: INT32 UNCOMPRESSED DO:0 FPO:4 SZ:843/843/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN
> name: BINARY UNCOMPRESSED DO:0 FPO:847 SZ:3214/3214/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
> age: INT32 UNCOMPRESSED DO:0 FPO:4061 SZ:438/438/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
> registration: BINARY UNCOMPRESSED DO:0 FPO:4499 SZ:241/241/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
> contributions: FLOAT UNCOMPRESSED DO:0 FPO:4740 SZ:843/843/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN
> voterzone: INT32 UNCOMPRESSED DO:0 FPO:5583 SZ:843/843/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN
> create_timestamp: INT96 UNCOMPRESSED DO:0 FPO:6426 SZ:2642/2642/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
> create_date: INT32 UNCOMPRESSED DO:0 FPO:9068 SZ:838/838/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN
> {code}
> I attached the log file and the data file
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)