[jira] [Commented] (DRILL-817) Parquet file created from impala using data from an impala table causes drill to hang

["Jason Altekruse (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016934#comment-14016934 ] 

Jason Altekruse commented on DRILL-817:
---------------------------------------

I have tried reading files created in this manner from Impala like this. Is the file you generated very large? Could you post the file here for debugging? if its greater than 10mb you can just e-mail  me at altekrusejason@gmail.com

> Parquet file created from impala using data from an impala table causes drill to hang
> -------------------------------------------------------------------------------------
>
>                 Key: DRILL-817
>                 URL: https://issues.apache.org/jira/browse/DRILL-817
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Norris Lee
>            Assignee: Jason Altekruse
>
> A normal table was created with impala:
> {code}
> [localhost.localdomain:21000] > create table integer_table2 (keycolumn string, column1 int) row format delimited fields terminated by ',';
> Query: create table integer_table2 (keycolumn string, column1 int) row format delimited fields terminated by ','
> Returned 0 row(s) in 0.10s
> [localhost.localdomain:21000] > show tables;
> Query: show tables
> +-----------------------+
> | name                  |
> +-----------------------+
> | integer_table2        |
> +-----------------------+
> Returned 1 row(s) in 0.01s
> {code}
> Data from a csv is loaded into the table
> {code}
> 46:44 PDT 2014)
> [localhost.localdomain:21000] > load data inpath '/user/hive/warehouse/Integer_Table.csv' overwrite into table integer_table2;
> Query: load data inpath '/user/hive/warehouse/Integer_Table.csv' overwrite into table integer_table2
> +----------------------------------------------------------+
> | summary                                                  |
> +----------------------------------------------------------+
> | Loaded 1 file(s). Total files in destination location: 1 |
> +----------------------------------------------------------+
> Returned 1 row(s) in 0.23s
> {code}
> A parquet table was then created in impala and data loaded in from the first table:
> {code}
> [localhost.localdomain:21000] > create table integer_table (keycolumn string, column1 int) row format delimited fields terminated by ',' stored as parquet;
> Query: create table integer_table (keycolumn string, column1 int) row format delimited fields terminated by ',' stored as parquet
> Returned 0 row(s) in 0.10s
> [localhost.localdomain:21000] > insert overwrite table integer_table select * from integer_table2;
> Query: insert overwrite table integer_table select * from integer_table2
> Inserted 18 rows in 0.20s
> {code}
> The parquet file (in the format <long alphanumeric string>_data.0 and renamed) was extracted from hdfs and moved to drill's dfs. The query through sqlline hangs:
> {code}
> 0: jdbc:drill:schema=hivestg> select * from `dfs`.`/opt/drill/integer.parquet`;
> <hang>
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)