You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sunny Jain <su...@gmail.com> on 2020/03/10 06:19:51 UTC

Exclusive locks acquired at the hive table has not got released automatically when hive services gets restarted

Hi Everyone,

While running Insert command or Alter table drop partition command, if hive
services got restarted in between, then that acquired the exclusive lock on
that partition which will remains after the restart also and for that kind
of job there is no yarn application id is generated sometimes and if
generated then it also got succeeded but exclusive locks remains on that
table or partition, which later we have to manually released from the table
or partitioned.

So why these locks remains on that partition or table and how these kind of
scenarios can be handle at our end?

Is there any workaround for these kind of scenarios?

Kindly help on this issue as I have not found any conclusive thing on this
part over the internet.

Thanks & Regards
Sunny Jain

Re: Exclusive locks acquired at the hive table has not got released automatically when hive services gets restarted

Posted by Peter Vary <pv...@cloudera.com>.
Hi,

Quick questions:
- Which version of Hive are you using?
- What type of LockManager is used?

ZKLockManager is using ephemeral nodes to remove stale locks, but the
default timeout is a bit high.
DBLockManager uses heartbeat to identify stale locks, but you need a
AcidHouseKeeperService to clean them up.

Thanks, Peter

Sunny Jain <su...@gmail.com> ezt írta (időpont: 2020. márc. 10.,
Ke 20:18):

> Hi Everyone,
>
> While running Insert command or Alter table drop partition command, if hive
> services got restarted in between, then that acquired the exclusive lock on
> that partition which will remains after the restart also and for that kind
> of job there is no yarn application id is generated sometimes and if
> generated then it also got succeeded but exclusive locks remains on that
> table or partition, which later we have to manually released from the table
> or partitioned.
>
> So why these locks remains on that partition or table and how these kind of
> scenarios can be handle at our end?
>
> Is there any workaround for these kind of scenarios?
>
> Kindly help on this issue as I have not found any conclusive thing on this
> part over the internet.
>
> Thanks & Regards
> Sunny Jain
>

Re: Exclusive locks acquired at the hive table has not got released automatically when hive services gets restarted

Posted by Peter Vary <pv...@cloudera.com.INVALID>.
Hi,

Quick questions:
- Which version of Hive are you using?
- What type of LockManager is used?

ZKLockManager is using ephemeral nodes to remove stale locks, but the
default timeout is a bit high.
DBLockManager uses heartbeat to identify stale locks, but you need a
AcidHouseKeeperService to clean them up.

Thanks, Peter

Sunny Jain <su...@gmail.com> ezt írta (időpont: 2020. márc. 10.,
Ke 20:18):

> Hi Everyone,
>
> While running Insert command or Alter table drop partition command, if hive
> services got restarted in between, then that acquired the exclusive lock on
> that partition which will remains after the restart also and for that kind
> of job there is no yarn application id is generated sometimes and if
> generated then it also got succeeded but exclusive locks remains on that
> table or partition, which later we have to manually released from the table
> or partitioned.
>
> So why these locks remains on that partition or table and how these kind of
> scenarios can be handle at our end?
>
> Is there any workaround for these kind of scenarios?
>
> Kindly help on this issue as I have not found any conclusive thing on this
> part over the internet.
>
> Thanks & Regards
> Sunny Jain
>