You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by James Warren <ja...@stanfordalumni.org> on 2013/02/05 21:21:11 UTC

unbalanced transaction calls

As part of our daily workflow, we're running a few hundred hive
queries that are coordinated through oozie.  Recently we're
encountering issues where on average a job or two fails - and never
the same query.  The observed error is:

FAILED: Error in metadata: java.lang.RuntimeException:
commitTransaction was called but openTransactionCalls = 0. This
probably indicates that there are unbalanced calls to
openTransaction/commitTransaction

which I've seen was referred to in HIVE-1760 but patched in 0.7.

We're running Hive 0.9 (from CDH 4.1 - will redirect to the Cloudera
lists if that is a more appropriate form) - any ideas / suggestions
where I should start looking?

cheers,
-James

Re: unbalanced transaction calls

Posted by Hemanth Yamijala <yh...@gmail.com>.

Thanks, Vinod. We'll try and find this out.


On Sat, Feb 23, 2013 at 3:25 AM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:

> If we can enable access logs on the meta store and figure out the last
> query before the one that failed, it should be easy to track it.
>
> I did find a couple of unbalanced calls in code. Will look more carefully
> and file a ticket.
>
> Thanks,
> +Vinod Kumar Vavilapalli
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Feb 22, 2013, at 4:45 AM, Hemanth Yamijala wrote:
>
> Hi,
>
> This error is coming when we are running a CREATE TABLE script. Where are
> you suggesting we make these changes ?
>
>
> On Fri, Feb 22, 2013 at 1:31 PM, Navis류승우 <na...@nexr.com> wrote:
>
>> Could you try to make transaction related methods synchronized?
>>
>> It's
>> openTransaction/commitTransaction/isActiveTransaction/rollbackTransaction
>> on ObjectStore.java
>>
>> 2013/2/22 Hemanth Yamijala <yh...@gmail.com>:
>> > Hi,
>> >
>> > We are running into the same problem as well. Is there any clue what
>> could
>> > be wrong ?
>> >
>> > Thanks
>> > hemanth
>> >
>> >
>> > On Wed, Feb 6, 2013 at 1:51 AM, James Warren
>> > <ja...@stanfordalumni.org> wrote:
>> >>
>> >> As part of our daily workflow, we're running a few hundred hive
>> >> queries that are coordinated through oozie.  Recently we're
>> >> encountering issues where on average a job or two fails - and never
>> >> the same query.  The observed error is:
>> >>
>> >> FAILED: Error in metadata: java.lang.RuntimeException:
>> >> commitTransaction was called but openTransactionCalls = 0. This
>> >> probably indicates that there are unbalanced calls to
>> >> openTransaction/commitTransaction
>> >>
>> >> which I've seen was referred to in HIVE-1760 but patched in 0.7.
>> >>
>> >> We're running Hive 0.9 (from CDH 4.1 - will redirect to the Cloudera
>> >> lists if that is a more appropriate form) - any ideas / suggestions
>> >> where I should start looking?
>> >>
>> >> cheers,
>> >> -James
>> >
>> >
>>
>
>
>

Re: unbalanced transaction calls

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

If we can enable access logs on the meta store and figure out the last query before the one that failed, it should be easy to track it.

I did find a couple of unbalanced calls in code. Will look more carefully and file a ticket.

Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Feb 22, 2013, at 4:45 AM, Hemanth Yamijala wrote:

> Hi,
> 
> This error is coming when we are running a CREATE TABLE script. Where are you suggesting we make these changes ?
> 
> 
> On Fri, Feb 22, 2013 at 1:31 PM, Navis류승우 <na...@nexr.com> wrote:
> Could you try to make transaction related methods synchronized?
> 
> It's openTransaction/commitTransaction/isActiveTransaction/rollbackTransaction
> on ObjectStore.java
> 
> 2013/2/22 Hemanth Yamijala <yh...@gmail.com>:
> > Hi,
> >
> > We are running into the same problem as well. Is there any clue what could
> > be wrong ?
> >
> > Thanks
> > hemanth
> >
> >
> > On Wed, Feb 6, 2013 at 1:51 AM, James Warren
> > <ja...@stanfordalumni.org> wrote:
> >>
> >> As part of our daily workflow, we're running a few hundred hive
> >> queries that are coordinated through oozie.  Recently we're
> >> encountering issues where on average a job or two fails - and never
> >> the same query.  The observed error is:
> >>
> >> FAILED: Error in metadata: java.lang.RuntimeException:
> >> commitTransaction was called but openTransactionCalls = 0. This
> >> probably indicates that there are unbalanced calls to
> >> openTransaction/commitTransaction
> >>
> >> which I've seen was referred to in HIVE-1760 but patched in 0.7.
> >>
> >> We're running Hive 0.9 (from CDH 4.1 - will redirect to the Cloudera
> >> lists if that is a more appropriate form) - any ideas / suggestions
> >> where I should start looking?
> >>
> >> cheers,
> >> -James
> >
> >
>

Re: unbalanced transaction calls

Posted by Hemanth Yamijala <yh...@gmail.com>.

Hi,

This error is coming when we are running a CREATE TABLE script. Where are
you suggesting we make these changes ?


On Fri, Feb 22, 2013 at 1:31 PM, Navis류승우 <na...@nexr.com> wrote:

> Could you try to make transaction related methods synchronized?
>
> It's
> openTransaction/commitTransaction/isActiveTransaction/rollbackTransaction
> on ObjectStore.java
>
> 2013/2/22 Hemanth Yamijala <yh...@gmail.com>:
> > Hi,
> >
> > We are running into the same problem as well. Is there any clue what
> could
> > be wrong ?
> >
> > Thanks
> > hemanth
> >
> >
> > On Wed, Feb 6, 2013 at 1:51 AM, James Warren
> > <ja...@stanfordalumni.org> wrote:
> >>
> >> As part of our daily workflow, we're running a few hundred hive
> >> queries that are coordinated through oozie.  Recently we're
> >> encountering issues where on average a job or two fails - and never
> >> the same query.  The observed error is:
> >>
> >> FAILED: Error in metadata: java.lang.RuntimeException:
> >> commitTransaction was called but openTransactionCalls = 0. This
> >> probably indicates that there are unbalanced calls to
> >> openTransaction/commitTransaction
> >>
> >> which I've seen was referred to in HIVE-1760 but patched in 0.7.
> >>
> >> We're running Hive 0.9 (from CDH 4.1 - will redirect to the Cloudera
> >> lists if that is a more appropriate form) - any ideas / suggestions
> >> where I should start looking?
> >>
> >> cheers,
> >> -James
> >
> >
>

Re: unbalanced transaction calls

Posted by Navis류승우 <na...@nexr.com>.

Could you try to make transaction related methods synchronized?

It's openTransaction/commitTransaction/isActiveTransaction/rollbackTransaction
on ObjectStore.java

2013/2/22 Hemanth Yamijala <yh...@gmail.com>:
> Hi,
>
> We are running into the same problem as well. Is there any clue what could
> be wrong ?
>
> Thanks
> hemanth
>
>
> On Wed, Feb 6, 2013 at 1:51 AM, James Warren
> <ja...@stanfordalumni.org> wrote:
>>
>> As part of our daily workflow, we're running a few hundred hive
>> queries that are coordinated through oozie.  Recently we're
>> encountering issues where on average a job or two fails - and never
>> the same query.  The observed error is:
>>
>> FAILED: Error in metadata: java.lang.RuntimeException:
>> commitTransaction was called but openTransactionCalls = 0. This
>> probably indicates that there are unbalanced calls to
>> openTransaction/commitTransaction
>>
>> which I've seen was referred to in HIVE-1760 but patched in 0.7.
>>
>> We're running Hive 0.9 (from CDH 4.1 - will redirect to the Cloudera
>> lists if that is a more appropriate form) - any ideas / suggestions
>> where I should start looking?
>>
>> cheers,
>> -James
>
>

Re: unbalanced transaction calls

Posted by Hemanth Yamijala <yh...@gmail.com>.

Hi,

We are running into the same problem as well. Is there any clue what could
be wrong ?

Thanks
hemanth


On Wed, Feb 6, 2013 at 1:51 AM, James Warren <
james.warren@stanfordalumni.org> wrote:

> As part of our daily workflow, we're running a few hundred hive
> queries that are coordinated through oozie.  Recently we're
> encountering issues where on average a job or two fails - and never
> the same query.  The observed error is:
>
> FAILED: Error in metadata: java.lang.RuntimeException:
> commitTransaction was called but openTransactionCalls = 0. This
> probably indicates that there are unbalanced calls to
> openTransaction/commitTransaction
>
> which I've seen was referred to in HIVE-1760 but patched in 0.7.
>
> We're running Hive 0.9 (from CDH 4.1 - will redirect to the Cloudera
> lists if that is a more appropriate form) - any ideas / suggestions
> where I should start looking?
>
> cheers,
> -James
>