You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Mich Talebzadeh <mi...@gmail.com> on 2016/08/01 21:50:58 UTC
How can I force Hive to start compaction on a table immediately
Rather than queuing it
hive> alter table payees COMPACT 'major';
Compaction enqueued.
OK
Thanks
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
Re: How can I force Hive to start compaction on a table immediately
Posted by Mich Talebzadeh <mi...@gmail.com>.
Thanks Alan.
One crude solution would be to copy data from the ACID table to a simple
table and present that table to Spark to see the data.
This is basically Spark optimiser issue not the engine itself
My Hive runs on Spark query engine and all works fine there.
HTH
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On 1 August 2016 at 23:47, Alan Gates <al...@gmail.com> wrote:
> There’s no way to force immediate compaction. If there are compaction
> workers in the metastore that aren’t busy they should pick that up
> immediately. But there isn’t an ability to create a worker thread and
> start compacting.
>
> Alan.
>
> > On Aug 1, 2016, at 14:50, Mich Talebzadeh <mi...@gmail.com>
> wrote:
> >
> >
> > Rather than queuing it
> >
> > hive> alter table payees COMPACT 'major';
> > Compaction enqueued.
> > OK
> >
> > Thanks
> >
> > Dr Mich Talebzadeh
> >
> > LinkedIn
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >
> > http://talebzadehmich.wordpress.com
> >
> > Disclaimer: Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
> >
>
>
Re: How can I force Hive to start compaction on a table immediately
Posted by Alan Gates <al...@gmail.com>.
There’s no way to force immediate compaction. If there are compaction workers in the metastore that aren’t busy they should pick that up immediately. But there isn’t an ability to create a worker thread and start compacting.
Alan.
> On Aug 1, 2016, at 14:50, Mich Talebzadeh <mi...@gmail.com> wrote:
>
>
> Rather than queuing it
>
> hive> alter table payees COMPACT 'major';
> Compaction enqueued.
> OK
>
> Thanks
>
> Dr Mich Talebzadeh
>
> LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>
> http://talebzadehmich.wordpress.com
>
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>