You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Amareshwari Sri Ramadasu <am...@yahoo-inc.com> on 2010/10/01 06:25:58 UTC
Create table as select fails with error "Error in metadata:
org.apache.thrift.TApplicationException"
Hi,
Create table as select queries fail with org.apache.thrift.TApplicationException in our clusters for some queries.
Following is the stack trace for the exception :
Error in metadata: org.apache.thrift.TApplicationException: Internal error processing create_table
org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.TApplicationException: Internal error processing
create_table
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:405)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2465)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:180)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:895)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:764)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:640)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: Internal error processing create_table
at org.apache.thrift.TApplicationException.read(TApplicationException.java:107)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:566)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:549)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:281)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:399)
When we tried with hive.metastore.connect.retries=10, it fails though it succeeds occasionally.
The metastore logs have the following exception
Internal error processing create_table
java.lang.RuntimeException: Commit is called, but transaction is not active. Either there are mismatching open and
close calls or rollback was called in the same trasaction
at org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:250)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:795)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$600(HiveMetaStore.java:79)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:816)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:813)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:234)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:813)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table.process(ThriftHiveMetastore.java:1992)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor.process(ThriftHiveMetastore.java:1644)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Can somebody help us find the root cause of the problem?
Thanks
Amareshwari
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by "S. Venkatesh" <ve...@innerzeal.com>.
Hi Carl,
Awesome, we'll try this and let you know.
Sincerely appreciated,
Venkatesh
On Fri, Oct 1, 2010 at 2:21 PM, Carl Steinbach <ca...@cloudera.com> wrote:
> Hi Venkatesh,
> I attached an interim patch to HIVE-1681. Please try applying this and let
> me know if it fixes your problem.
> Thanks.
> Carl
>
> On Fri, Oct 1, 2010 at 1:31 AM, Carl Steinbach <ca...@cloudera.com> wrote:
>>
>> Hi Venkatesh,
>> I filed HIVE-1679 which covers the issue I described earlier as well
>> as HIVE-1681 which I think is the real root cause of the problem you
>> are seeing. Please see HIVE-1681 for more information.
>> Thanks.
>> Carl
>>
>> On Fri, Oct 1, 2010 at 12:51 AM, S. Venkatesh <ve...@innerzeal.com>
>> wrote:
>>>
>>> We are seeing this exception:
>>>
>>> Internal error processing create_table
>>> java.lang.RuntimeException: Commit is called, but transaction is not
>>> active.
>>> Either there are mismatching open and close calls or rollback was called
>>> in the
>>> same transaction
>>>
>>> Carl, this suggests that the transaction is not active when commit was
>>> called. I concur with your observation.
>>>
>>> Venkatesh
>>>
>>> On Fri, Oct 1, 2010 at 12:21 PM, Carl Steinbach <ca...@cloudera.com>
>>> wrote:
>>> > It looks like the root of the problem is that the code in ObjectStore
>>> > is not
>>> > detecting failed transactions, and hence is not rolling them back.
>>> > Using JDO with locally managed transactions, you're expected to do
>>> > something
>>> > like this:
>>> > Transaction tx = pm.currentTransaction();
>>> > try {
>>> > tx.begin();
>>> > { do some stuff }
>>> > tx.commit();
>>> > } finally {
>>> > if (tx.isActive()) {
>>> > tx.rollback();
>>> > }
>>> > }
>>> > But the code in ObjectStore instead wraps these methods in
>>> > openTransaction()
>>> > and commitTransaction(). After calling tx.commit() the method
>>> > commitTransaction() should then call tx.isActive() to check if the
>>> > transaction failed and call rollback if appropriate. Currently it
>>> > doesn't do
>>> > this and instead always returns false.
>>> > I'm filing a ticket against this now.
>>> > Thanks.
>>> > Carl
>>> >
>>
>
>
--
Regards,
Venkatesh
“Perfection (in design) is achieved not when there is nothing more to
add, but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by Carl Steinbach <ca...@cloudera.com>.
Hi Venkatesh,
I attached an interim patch to HIVE-1681. Please try applying this and let
me know if it fixes your problem.
Thanks.
Carl
On Fri, Oct 1, 2010 at 1:31 AM, Carl Steinbach <ca...@cloudera.com> wrote:
> Hi Venkatesh,
>
> I filed HIVE-1679 which covers the issue I described earlier as well
> as HIVE-1681 which I think is the real root cause of the problem you
> are seeing. Please see HIVE-1681 for more information.
>
> Thanks.
>
> Carl
>
>
> On Fri, Oct 1, 2010 at 12:51 AM, S. Venkatesh <ve...@innerzeal.com>wrote:
>
>> We are seeing this exception:
>>
>> Internal error processing create_table
>> java.lang.RuntimeException: Commit is called, but transaction is not
>> active.
>> Either there are mismatching open and close calls or rollback was called
>> in the
>> same transaction
>>
>> Carl, this suggests that the transaction is not active when commit was
>> called. I concur with your observation.
>>
>> Venkatesh
>>
>> On Fri, Oct 1, 2010 at 12:21 PM, Carl Steinbach <ca...@cloudera.com>
>> wrote:
>> > It looks like the root of the problem is that the code in ObjectStore is
>> not
>> > detecting failed transactions, and hence is not rolling them back.
>> > Using JDO with locally managed transactions, you're expected to do
>> something
>> > like this:
>> > Transaction tx = pm.currentTransaction();
>> > try {
>> > tx.begin();
>> > { do some stuff }
>> > tx.commit();
>> > } finally {
>> > if (tx.isActive()) {
>> > tx.rollback();
>> > }
>> > }
>> > But the code in ObjectStore instead wraps these methods in
>> openTransaction()
>> > and commitTransaction(). After calling tx.commit() the method
>> > commitTransaction() should then call tx.isActive() to check if the
>> > transaction failed and call rollback if appropriate. Currently it
>> doesn't do
>> > this and instead always returns false.
>> > I'm filing a ticket against this now.
>> > Thanks.
>> > Carl
>> >
>>
>
>
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by Carl Steinbach <ca...@cloudera.com>.
Hi Venkatesh,
I filed HIVE-1679 which covers the issue I described earlier as well
as HIVE-1681 which I think is the real root cause of the problem you
are seeing. Please see HIVE-1681 for more information.
Thanks.
Carl
On Fri, Oct 1, 2010 at 12:51 AM, S. Venkatesh <ve...@innerzeal.com>wrote:
> We are seeing this exception:
>
> Internal error processing create_table
> java.lang.RuntimeException: Commit is called, but transaction is not
> active.
> Either there are mismatching open and close calls or rollback was called in
> the
> same transaction
>
> Carl, this suggests that the transaction is not active when commit was
> called. I concur with your observation.
>
> Venkatesh
>
> On Fri, Oct 1, 2010 at 12:21 PM, Carl Steinbach <ca...@cloudera.com> wrote:
> > It looks like the root of the problem is that the code in ObjectStore is
> not
> > detecting failed transactions, and hence is not rolling them back.
> > Using JDO with locally managed transactions, you're expected to do
> something
> > like this:
> > Transaction tx = pm.currentTransaction();
> > try {
> > tx.begin();
> > { do some stuff }
> > tx.commit();
> > } finally {
> > if (tx.isActive()) {
> > tx.rollback();
> > }
> > }
> > But the code in ObjectStore instead wraps these methods in
> openTransaction()
> > and commitTransaction(). After calling tx.commit() the method
> > commitTransaction() should then call tx.isActive() to check if the
> > transaction failed and call rollback if appropriate. Currently it doesn't
> do
> > this and instead always returns false.
> > I'm filing a ticket against this now.
> > Thanks.
> > Carl
> >
>
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by "S. Venkatesh" <ve...@innerzeal.com>.
We are seeing this exception:
Internal error processing create_table
java.lang.RuntimeException: Commit is called, but transaction is not active.
Either there are mismatching open and close calls or rollback was called in the
same transaction
Carl, this suggests that the transaction is not active when commit was
called. I concur with your observation.
Venkatesh
On Fri, Oct 1, 2010 at 12:21 PM, Carl Steinbach <ca...@cloudera.com> wrote:
> It looks like the root of the problem is that the code in ObjectStore is not
> detecting failed transactions, and hence is not rolling them back.
> Using JDO with locally managed transactions, you're expected to do something
> like this:
> Transaction tx = pm.currentTransaction();
> try {
> tx.begin();
> { do some stuff }
> tx.commit();
> } finally {
> if (tx.isActive()) {
> tx.rollback();
> }
> }
> But the code in ObjectStore instead wraps these methods in openTransaction()
> and commitTransaction(). After calling tx.commit() the method
> commitTransaction() should then call tx.isActive() to check if the
> transaction failed and call rollback if appropriate. Currently it doesn't do
> this and instead always returns false.
> I'm filing a ticket against this now.
> Thanks.
> Carl
>
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by Carl Steinbach <ca...@cloudera.com>.
It looks like the root of the problem is that the code in ObjectStore is not
detecting failed transactions, and hence is not rolling them back.
Using JDO with locally managed transactions, you're expected to do something
like this:
Transaction tx = pm.currentTransaction();
try {
tx.begin();
{ do some stuff }
tx.commit();
} finally {
if (tx.isActive()) {
tx.rollback();
}
}
But the code in ObjectStore instead wraps these methods in openTransaction()
and commitTransaction(). After calling tx.commit() the method
commitTransaction() should then call tx.isActive() to check if the
transaction failed and call rollback if appropriate. Currently it doesn't do
this and instead always returns false.
I'm filing a ticket against this now.
Thanks.
Carl
Hive 0.5 Unit TestParse Test Case Failures
Posted by Stephen Watt <sw...@us.ibm.com>.
Hi Folks
I'm working on getting Hive 0.5 working on Apache Harmony and IBM Java 6.
I've almost got it passing everything running "ant clean test tar", but at
least half the asserts in the TestParse TestCase are failing with the
AssertionFailedError's of "Unexpected Exception" and "Semantic Analysis
has unexpected output with error code = 1". I've located the TestParse.xml
file, which has the JVM Properties used along with the System Out and
Error for the Testcase, which shows the errors I mentioned.
I've seen some other tickets with this exception, but they are resolved
already in 0.5. Clearly, this was all tested with SUN Java and this issue
does not show up with that JRE, but given that I'm trying to get it
working on alternate JREs, can anyone point me in a direction that might
help me get to the bottom of the issue?
junit.framework.AssertionFailedError: Semantic Analysis has unexpected
output with error code = 1
at junit.framework.Assert.fail(Assert.java:47)
at
org.apache.hadoop.hive.ql.parse.TestParse.testParse_case_sensitivity(TestParse.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:600)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
Regards
Steve Watt
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by "S. Venkatesh" <ve...@innerzeal.com>.
On Fri, Oct 1, 2010 at 12:24 PM, Ning Zhang <nz...@facebook.com> wrote:
> If your success rate is increased by increasing hive.metastore.connect.retries, mostly likely your Thrift server is too busy to get connected. Can you check your /tmp/<userid>/hive.log in your local machine and see if exception related to metastore?
Ning, the load on metastore is very low, its this one user executing
the CTAS. No one else is using this setup but QE.
Re: Create table as select fails with error "Error in metadata:
org.apache.thrift.TApplicationException"
Posted by Ning Zhang <nz...@facebook.com>.
If your success rate is increased by increasing hive.metastore.connect.retries, mostly likely your Thrift server is too busy to get connected. Can you check your /tmp/<userid>/hive.log in your local machine and see if exception related to metastore?
On Sep 30, 2010, at 11:21 PM, S. Venkatesh wrote:
>> Amareshwari, this should be a metastore issue. Did you see this kind of
>> issue in create table (without select) alone?
> Nope.
>
>> Which RDBMS are you using for metastore? MySQL or Oracle?
> MySQL
>
>> Was your
>> database/thrift server heavy-loaded at that time?
> Running with just one user.
>
> --
> Regards,
> Venkatesh
>
> “Perfection (in design) is achieved not when there is nothing more to
> add, but rather when there is nothing more to take away.”
> - Antoine de Saint-Exupéry
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by "S. Venkatesh" <ve...@innerzeal.com>.
> Amareshwari, this should be a metastore issue. Did you see this kind of
> issue in create table (without select) alone?
Nope.
> Which RDBMS are you using for metastore? MySQL or Oracle?
MySQL
> Was your
> database/thrift server heavy-loaded at that time?
Running with just one user.
--
Regards,
Venkatesh
“Perfection (in design) is achieved not when there is nothing more to
add, but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry
Re: Create table as select fails with error "Error in metadata:
org.apache.thrift.TApplicationException"
Posted by Amareshwari Sri Ramadasu <am...@yahoo-inc.com>.
No. We did not see any issue with create table (without select). Create table as select also passes sometimes.
We are using MySQL for our metastore.
Will get back to you on load at the time of create. Is there a direct way to know the load?
So, what is the solution/workaround if there is a metastore issue?
Thanks
Amareshwari
On 10/1/10 11:38 AM, "Ning Zhang" <nz...@facebook.com> wrote:
Amareshwari, this should be a metastore issue. Did you see this kind of issue in create table (without select) alone?
Which RDBMS are you using for metastore? MySQL or Oracle? Was your database/thrift server heavy-loaded at that time?
On Sep 30, 2010, at 10:28 PM, Amareshwari Sri Ramadasu wrote:
Carl, we are using trunk version.
On 10/1/10 10:16 AM, "Carl Steinbach" <carl@cloudera.com <x-...@cloudera.com> > wrote:
Hi Amareshwari,
Which version of Hive are you using to run the Hive metastore server?
Thanks.
Carl
On Thu, Sep 30, 2010 at 9:25 PM, Amareshwari Sri Ramadasu <amarsri@yahoo-inc.com <x-...@yahoo-inc.com> > wrote:
Hi,
Create table as select queries fail with org.apache.thrift.TApplicationException in our clusters for some queries.
Following is the stack trace for the exception :
Error in metadata: org.apache.thrift.TApplicationException: Internal error processing create_table
org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.TApplicationException: Internal error processing
create_table
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:405)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2465)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:180)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:895)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:764)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:640)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: Internal error processing create_table
at org.apache.thrift.TApplicationException.read(TApplicationException.java:107)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:566)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:549)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:281)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:399)
When we tried with hive.metastore.connect.retries=10, it fails though it succeeds occasionally.
The metastore logs have the following exception
Internal error processing create_table
java.lang.RuntimeException: Commit is called, but transaction is not active. Either there are mismatching open and
close calls or rollback was called in the same trasaction
at org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:250)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:795)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$600(HiveMetaStore.java:79)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:816)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:813)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:234)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:813)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table.process(ThriftHiveMetastore.java:1992)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor.process(ThriftHiveMetastore.java:1644)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Can somebody help us find the root cause of the problem?
Thanks
Amareshwari
Re: Create table as select fails with error "Error in metadata:
org.apache.thrift.TApplicationException"
Posted by Ning Zhang <nz...@facebook.com>.
Amareshwari, this should be a metastore issue. Did you see this kind of issue in create table (without select) alone?
Which RDBMS are you using for metastore? MySQL or Oracle? Was your database/thrift server heavy-loaded at that time?
On Sep 30, 2010, at 10:28 PM, Amareshwari Sri Ramadasu wrote:
Carl, we are using trunk version.
On 10/1/10 10:16 AM, "Carl Steinbach" <ca...@cloudera.com>> wrote:
Hi Amareshwari,
Which version of Hive are you using to run the Hive metastore server?
Thanks.
Carl
On Thu, Sep 30, 2010 at 9:25 PM, Amareshwari Sri Ramadasu <am...@yahoo-inc.com>> wrote:
Hi,
Create table as select queries fail with org.apache.thrift.TApplicationException in our clusters for some queries.
Following is the stack trace for the exception :
Error in metadata: org.apache.thrift.TApplicationException: Internal error processing create_table
org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.TApplicationException: Internal error processing
create_table
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:405)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2465)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:180)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:895)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:764)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:640)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: Internal error processing create_table
at org.apache.thrift.TApplicationException.read(TApplicationException.java:107)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:566)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:549)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:281)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:399)
When we tried with hive.metastore.connect.retries=10, it fails though it succeeds occasionally.
The metastore logs have the following exception
Internal error processing create_table
java.lang.RuntimeException: Commit is called, but transaction is not active. Either there are mismatching open and
close calls or rollback was called in the same trasaction
at org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:250)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:795)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$600(HiveMetaStore.java:79)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:816)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:813)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:234)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:813)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table.process(ThriftHiveMetastore.java:1992)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor.process(ThriftHiveMetastore.java:1644)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Can somebody help us find the root cause of the problem?
Thanks
Amareshwari
Re: Create table as select fails with error "Error in metadata:
org.apache.thrift.TApplicationException"
Posted by Amareshwari Sri Ramadasu <am...@yahoo-inc.com>.
Carl, we are using trunk version.
On 10/1/10 10:16 AM, "Carl Steinbach" <ca...@cloudera.com> wrote:
Hi Amareshwari,
Which version of Hive are you using to run the Hive metastore server?
Thanks.
Carl
On Thu, Sep 30, 2010 at 9:25 PM, Amareshwari Sri Ramadasu <am...@yahoo-inc.com> wrote:
Hi,
Create table as select queries fail with org.apache.thrift.TApplicationException in our clusters for some queries.
Following is the stack trace for the exception :
Error in metadata: org.apache.thrift.TApplicationException: Internal error processing create_table
org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.TApplicationException: Internal error processing
create_table
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:405)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2465)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:180)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:895)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:764)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:640)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: Internal error processing create_table
at org.apache.thrift.TApplicationException.read(TApplicationException.java:107)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:566)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:549)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:281)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:399)
When we tried with hive.metastore.connect.retries=10, it fails though it succeeds occasionally.
The metastore logs have the following exception
Internal error processing create_table
java.lang.RuntimeException: Commit is called, but transaction is not active. Either there are mismatching open and
close calls or rollback was called in the same trasaction
at org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:250)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:795)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$600(HiveMetaStore.java:79)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:816)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:813)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:234)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:813)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table.process(ThriftHiveMetastore.java:1992)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor.process(ThriftHiveMetastore.java:1644)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Can somebody help us find the root cause of the problem?
Thanks
Amareshwari
Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"
Posted by Carl Steinbach <ca...@cloudera.com>.
Hi Amareshwari,
Which version of Hive are you using to run the Hive metastore server?
Thanks.
Carl
On Thu, Sep 30, 2010 at 9:25 PM, Amareshwari Sri Ramadasu <
amarsri@yahoo-inc.com> wrote:
> Hi,
>
> Create table as select queries fail with
> org.apache.thrift.TApplicationException in our clusters for some queries.
>
> Following is the stack trace for the exception :
> Error in metadata: org.apache.thrift.TApplicationException: Internal error
> processing create_table
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.thrift.TApplicationException: Internal error processing
> create_table
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:405)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2465)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:180)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:895)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:764)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:640)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.apache.thrift.TApplicationException: Internal error
> processing create_table
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:107)
> at
>
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:566)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:549)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:281)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:399)
>
> When we tried with hive.metastore.connect.retries=10, it fails though it
> succeeds occasionally.
>
> The metastore logs have the following exception
> Internal error processing create_table
> java.lang.RuntimeException: Commit is called, but transaction is not
> active. Either there are mismatching open and
> close calls or rollback was called in the same trasaction
> at
> org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:250)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:795)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$600(HiveMetaStore.java:79)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:816)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$13.run(HiveMetaStore.java:813)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:234)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:813)
> at
>
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table.process(ThriftHiveMetastore.java:1992)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor.process(ThriftHiveMetastore.java:1644)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
> Can somebody help us find the root cause of the problem?
>
> Thanks
> Amareshwari
>
>