You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Gopal Vijayaraghavan <go...@apache.org> on 2018/07/25 05:45:16 UTC
Re: Total length of orc clustered table is always 2^31 in
TezSplitGrouper
> Search ’Total length’ in log sys_dag_xxx, it is 2147483648.
This is the INT_MAX “placeholder” value for uncompacted ACID tables.
This is because with ACIDv1 there is no way to generate splits against uncompacted files, so this gets “an empty bucket + unknown number of inserts + updates” placeholder value.
Cheers,
Gopal
Re: Total length of orc clustered table is always 2^31 in
TezSplitGrouper
Posted by 何宝宁 <ba...@ecreditpal.com>.
Thank you Gopal for pointing the root cause. After running command alter table xxx compact ‘major’ to request a force compaction, total length is right !
Is there any way to do compact immediately after insert values.
Bob He
Thanks
On 25 Jul 2018, at 1:45 PM, Gopal Vijayaraghavan <go...@apache.org> wrote:
> Search ’Total length’ in log sys_dag_xxx, it is 2147483648.
This is the INT_MAX “placeholder” value for uncompacted ACID tables.
This is because with ACIDv1 there is no way to generate splits against uncompacted files, so this gets “an empty bucket + unknown number of inserts + updates” placeholder value.
Cheers,
Gopal