[jira] [Resolved] (IMPALA-2807) During insert operation impala creates too many files for a table size < block size

["Tim Armstrong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-2807.
-----------------------------------
    Resolution: Won't Fix

IMPALA-8125 added an option to control this. Maybe we could be smarter about picking parallelism to trade-off file size, but I think we need a clearer problem to solve.

> During insert operation impala creates too many files for a table size < block size
> -----------------------------------------------------------------------------------
>
>                 Key: IMPALA-2807
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2807
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Perf Investigation
>    Affects Versions: Impala 2.3.0
>            Reporter: Dileep Kumar
>            Priority: Major
>
> When loading the "customer" table from TPC-DS based schema, total no. of files created is 20 (which is equal to number of impala nodes in the cluster).
> The total size of the this table is 204.2 MiB which can fit in a single block while it occupies 20 blocks in this case.
> When ran the same insert command with a single impalad running in the cluster single block was able to hold all the table data and only one hdfs file was created.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)