You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tajo.apache.org by Thanh Mai <ma...@gmail.com> on 2014/11/25 04:12:13 UTC
Tajo with TPC-H benchmark
Hi everyone,
I want to find the performance results of Tajo with the TPC-H benchmark in
both power test and throughput test (and thus, a sample QphH value for
reference). However, from the slides in Tajo website, I found only the
response times of some queries.
Have any of you tested Tajo in the TPC-H benchmark's throughput test or in
the multi-user situation ? If yes, could you share your experience ?
When I test Tajo with the TPC-H benchmark's throughput test, some queries
failed occasionally. I use Hive catalog instead of Tajo catalog. It seems
that Tajo has a problem when getting information from Hive catalog
concurrently for multiple queries running in parallel. Have any of you
experienced a similar situation ? Could you share your experience and
insights ?
Thank you very much!
Sincerely,
Mai Hai Thanh
Re: Tajo with TPC-H benchmark
Posted by Thanh Mai <ma...@gmail.com>.
Thank Hyunsik,
I tried your workaround. However, the problem still remains the same.
Sincerely,
On Fri, Nov 28, 2014 at 2:45 PM, Hyunsik Choi <hy...@apache.org> wrote:
> I just heard some information from Jinho. He experienced such a
> problem. If you set some config to enable Tajo catalog to access Hive
> Meta store, it uses thrift server of Hive. According to his advice, it
> causes such a problem frequently. This is a known problem in hive, but
> there is a workaround.
>
> The workaround manner is to copy 'hive-site.xml' of your hive to
> ${TAJO_HOME}/conf. Then, Tajo catalog directly accesses the persistent
> RDBMS being used by Hive instead of connecting Hive meta store via
> Thrift.
>
> Probably, It will solve your problem.
>
> Best regards,
> Hyunsik
>
> On Fri, Nov 28, 2014 at 1:10 PM, Thanh Mai <ma...@gmail.com> wrote:
> > Hi Hyunsik,
> >
> > I have already modified TPC-H queries to be able to run with Tajo. If the
> > queries are issued sequentially, there is no problem. However, if they
> are
> > issued concurrently, some queries will fail sometimes (not always).
> >
> > I cannot share the log file. Instead, I can share the error messages
> below.
> > It seems that the problem is not directly related to Tajo, but to Thrift
> or
> > ProtoBuf. I have also increased the value of
> > "hive.metastore.client.socket.timeout" from 20 to 200 and
> > "hive.server.read.socket.timeout" from 10 to 100 in hive-site.xml, but
> the
> > problem remains.
> >
> > Thank you for your help!
> >
> > Sincerely,
> >
> > Mai Hai Thanh
> >
> >
> ----------------------------------------------------------------------------------------------------------------------------------------------
> >
> > 2014-11-28 11:09:38,535 ERROR hive.log: Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out
> > org.apache.thrift.transport.TTransportException:
> > java.net.SocketTimeoutException: Read timed out
> > at
> >
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
> > at
> > org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> > at
> >
> org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:354)
> > at
> >
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
> > at
> > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> > at
> >
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_all_databases(ThriftHiveMetastore.java:625)
> > at
> >
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_all_databases(ThriftHiveMetastore.java:613)
> > at
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:837)
> > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
> > at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
> > at
> >
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
> > at
> >
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
> > at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
> > at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
> > at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> > at
> >
> org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
> > at
> >
> org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
> > at
> >
> org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
> > at
> >
> org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
> > at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> > at
> >
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> > at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> > at
> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> > at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> > at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> > at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> > at
> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> > at
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > at
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.net.SocketTimeoutException: Read timed out
> > at java.net.SocketInputStream.socketRead0(Native Method)
> > at java.net.SocketInputStream.read(SocketInputStream.java:152)
> > at java.net.SocketInputStream.read(SocketInputStream.java:122)
> > at
> java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
> > at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> > at
> >
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> > ... 47 more
> >
> > 2014-11-28 11:09:38,538 ERROR hive.log: Converting exception to
> > MetaException
> > 2014-11-28 11:09:38,550 ERROR org.apache.tajo.catalog.CatalogServer:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > 2014-11-28 11:09:38,550 ERROR
> > org.apache.tajo.catalog.AbstractCatalogClient:
> > com.google.protobuf.ServiceException:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > com.google.protobuf.ServiceException:
> com.google.protobuf.ServiceException:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
> > at
> >
> org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
> > at
> >
> org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
> > at
> >
> org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
> > at
> >
> org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
> > at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> > at
> >
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> > at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> > at
> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> > at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> > at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> > at
> >
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> > at
> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> > at
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > at
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: com.google.protobuf.ServiceException:
> > org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > at
> >
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:421)
> > at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
> > at
> >
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
> > at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> > ... 30 more
> > Caused by: org.apache.tajo.catalog.exception.CatalogException:
> > MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > at
> >
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:372)
> > at
> >
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
> > ... 33 more
> > Caused by: MetaException(message:Got exception:
> > org.apache.thrift.transport.TTransportException
> > java.net.SocketTimeoutException: Read timed out)
> > at
> >
> org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1102)
> > at
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:839)
> > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
> > at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
> > at
> >
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
> > ... 34 more
> >
> >
> > ERROR: Exception was thrown. Caused by
> > org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
> > "default" does not exist
> > Exception in thread "main" com.google.protobuf.ServiceException:
> > org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
> > "default" does not exist
> > at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
> > at
> > org.apache.tajo.client.TajoClient.getCurrentDatabase(TajoClient.java:262)
> > at org.apache.tajo.cli.TajoCli.<init>(TajoCli.java:243)
> > at org.apache.tajo.cli.TajoCli.main(TajoCli.java:690)
> > Caused by: org.apache.tajo.cli.InvalidClientSessionException: ERROR:
> > database "default" does not exist
> > at
> > org.apache.tajo.client.TajoClient.checkSessionAndGet(TajoClient.java:249)
> > at
> org.apache.tajo.client.TajoClient.access$000(TajoClient.java:75)
> > at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:265)
> > at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:262)
> > at
> > org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> > ... 3 more
> >
> >
> -------------------------------------------------------------------------------------------------------------------------------------------------
> >
> >
> > On Fri, Nov 28, 2014 at 10:36 AM, Hyunsik Choi <hy...@apache.org>
> wrote:
> >
> >> Hi Mai,
> >>
> >> Good to see you again!
> >>
> >> So far, we haven't performed QphH-based benchmark.
> >>
> >> Some of original TPC-H benchmark queries may not work in Tajo because
> >> scalar subquery, in-subquery, exist-subquery are not supported yet.
> >> So, you need to change slightly them into multiple queries.
> >>
> >> I haven't experienced such a situation you explained, could you share
> >> the error log that you met while you were executing multiple queries?
> >>
> >> Best regards,
> >> Hyunsik
> >>
> >>
> >>
> >> On Tue, Nov 25, 2014 at 12:12 PM, Thanh Mai <ma...@gmail.com>
> wrote:
> >> > Hi everyone,
> >> >
> >> > I want to find the performance results of Tajo with the TPC-H
> benchmark
> >> in
> >> > both power test and throughput test (and thus, a sample QphH value for
> >> > reference). However, from the slides in Tajo website, I found only the
> >> > response times of some queries.
> >> >
> >> > Have any of you tested Tajo in the TPC-H benchmark's throughput test
> or
> >> in
> >> > the multi-user situation ? If yes, could you share your experience ?
> >> >
> >> > When I test Tajo with the TPC-H benchmark's throughput test, some
> queries
> >> > failed occasionally. I use Hive catalog instead of Tajo catalog. It
> seems
> >> > that Tajo has a problem when getting information from Hive catalog
> >> > concurrently for multiple queries running in parallel. Have any of you
> >> > experienced a similar situation ? Could you share your experience and
> >> > insights ?
> >> >
> >> > Thank you very much!
> >> >
> >> > Sincerely,
> >> >
> >> > Mai Hai Thanh
> >>
>
Re: Tajo with TPC-H benchmark
Posted by Hyunsik Choi <hy...@apache.org>.
I just heard some information from Jinho. He experienced such a
problem. If you set some config to enable Tajo catalog to access Hive
Meta store, it uses thrift server of Hive. According to his advice, it
causes such a problem frequently. This is a known problem in hive, but
there is a workaround.
The workaround manner is to copy 'hive-site.xml' of your hive to
${TAJO_HOME}/conf. Then, Tajo catalog directly accesses the persistent
RDBMS being used by Hive instead of connecting Hive meta store via
Thrift.
Probably, It will solve your problem.
Best regards,
Hyunsik
On Fri, Nov 28, 2014 at 1:10 PM, Thanh Mai <ma...@gmail.com> wrote:
> Hi Hyunsik,
>
> I have already modified TPC-H queries to be able to run with Tajo. If the
> queries are issued sequentially, there is no problem. However, if they are
> issued concurrently, some queries will fail sometimes (not always).
>
> I cannot share the log file. Instead, I can share the error messages below.
> It seems that the problem is not directly related to Tajo, but to Thrift or
> ProtoBuf. I have also increased the value of
> "hive.metastore.client.socket.timeout" from 20 to 200 and
> "hive.server.read.socket.timeout" from 10 to 100 in hive-site.xml, but the
> problem remains.
>
> Thank you for your help!
>
> Sincerely,
>
> Mai Hai Thanh
>
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
> 2014-11-28 11:09:38,535 ERROR hive.log: Got exception:
> org.apache.thrift.transport.TTransportException
> java.net.SocketTimeoutException: Read timed out
> org.apache.thrift.transport.TTransportException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
> at
> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:354)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_all_databases(ThriftHiveMetastore.java:625)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_all_databases(ThriftHiveMetastore.java:613)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:837)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
> at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
> at
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
> at
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
> at
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
> at
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
> at
> org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> at
> org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
> at
> org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
> at
> org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
> at
> org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> ... 47 more
>
> 2014-11-28 11:09:38,538 ERROR hive.log: Converting exception to
> MetaException
> 2014-11-28 11:09:38,550 ERROR org.apache.tajo.catalog.CatalogServer:
> org.apache.tajo.catalog.exception.CatalogException:
> MetaException(message:Got exception:
> org.apache.thrift.transport.TTransportException
> java.net.SocketTimeoutException: Read timed out)
> 2014-11-28 11:09:38,550 ERROR
> org.apache.tajo.catalog.AbstractCatalogClient:
> com.google.protobuf.ServiceException:
> org.apache.tajo.catalog.exception.CatalogException:
> MetaException(message:Got exception:
> org.apache.thrift.transport.TTransportException
> java.net.SocketTimeoutException: Read timed out)
> com.google.protobuf.ServiceException: com.google.protobuf.ServiceException:
> org.apache.tajo.catalog.exception.CatalogException:
> MetaException(message:Got exception:
> org.apache.thrift.transport.TTransportException
> java.net.SocketTimeoutException: Read timed out)
> at
> org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
> at
> org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
> at
> org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
> at
> org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
> at
> org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.google.protobuf.ServiceException:
> org.apache.tajo.catalog.exception.CatalogException:
> MetaException(message:Got exception:
> org.apache.thrift.transport.TTransportException
> java.net.SocketTimeoutException: Read timed out)
> at
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:421)
> at
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
> at
> org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
> at
> org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> ... 30 more
> Caused by: org.apache.tajo.catalog.exception.CatalogException:
> MetaException(message:Got exception:
> org.apache.thrift.transport.TTransportException
> java.net.SocketTimeoutException: Read timed out)
> at
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:372)
> at
> org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
> ... 33 more
> Caused by: MetaException(message:Got exception:
> org.apache.thrift.transport.TTransportException
> java.net.SocketTimeoutException: Read timed out)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1102)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:839)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
> at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
> at
> org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
> ... 34 more
>
>
> ERROR: Exception was thrown. Caused by
> org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
> "default" does not exist
> Exception in thread "main" com.google.protobuf.ServiceException:
> org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
> "default" does not exist
> at
> org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
> at
> org.apache.tajo.client.TajoClient.getCurrentDatabase(TajoClient.java:262)
> at org.apache.tajo.cli.TajoCli.<init>(TajoCli.java:243)
> at org.apache.tajo.cli.TajoCli.main(TajoCli.java:690)
> Caused by: org.apache.tajo.cli.InvalidClientSessionException: ERROR:
> database "default" does not exist
> at
> org.apache.tajo.client.TajoClient.checkSessionAndGet(TajoClient.java:249)
> at org.apache.tajo.client.TajoClient.access$000(TajoClient.java:75)
> at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:265)
> at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:262)
> at
> org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
> ... 3 more
>
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
>
> On Fri, Nov 28, 2014 at 10:36 AM, Hyunsik Choi <hy...@apache.org> wrote:
>
>> Hi Mai,
>>
>> Good to see you again!
>>
>> So far, we haven't performed QphH-based benchmark.
>>
>> Some of original TPC-H benchmark queries may not work in Tajo because
>> scalar subquery, in-subquery, exist-subquery are not supported yet.
>> So, you need to change slightly them into multiple queries.
>>
>> I haven't experienced such a situation you explained, could you share
>> the error log that you met while you were executing multiple queries?
>>
>> Best regards,
>> Hyunsik
>>
>>
>>
>> On Tue, Nov 25, 2014 at 12:12 PM, Thanh Mai <ma...@gmail.com> wrote:
>> > Hi everyone,
>> >
>> > I want to find the performance results of Tajo with the TPC-H benchmark
>> in
>> > both power test and throughput test (and thus, a sample QphH value for
>> > reference). However, from the slides in Tajo website, I found only the
>> > response times of some queries.
>> >
>> > Have any of you tested Tajo in the TPC-H benchmark's throughput test or
>> in
>> > the multi-user situation ? If yes, could you share your experience ?
>> >
>> > When I test Tajo with the TPC-H benchmark's throughput test, some queries
>> > failed occasionally. I use Hive catalog instead of Tajo catalog. It seems
>> > that Tajo has a problem when getting information from Hive catalog
>> > concurrently for multiple queries running in parallel. Have any of you
>> > experienced a similar situation ? Could you share your experience and
>> > insights ?
>> >
>> > Thank you very much!
>> >
>> > Sincerely,
>> >
>> > Mai Hai Thanh
>>
Re: Tajo with TPC-H benchmark
Posted by Thanh Mai <ma...@gmail.com>.
Hi Hyunsik,
I have already modified TPC-H queries to be able to run with Tajo. If the
queries are issued sequentially, there is no problem. However, if they are
issued concurrently, some queries will fail sometimes (not always).
I cannot share the log file. Instead, I can share the error messages below.
It seems that the problem is not directly related to Tajo, but to Thrift or
ProtoBuf. I have also increased the value of
"hive.metastore.client.socket.timeout" from 20 to 200 and
"hive.server.read.socket.timeout" from 10 to 100 in hive-site.xml, but the
problem remains.
Thank you for your help!
Sincerely,
Mai Hai Thanh
----------------------------------------------------------------------------------------------------------------------------------------------
2014-11-28 11:09:38,535 ERROR hive.log: Got exception:
org.apache.thrift.transport.TTransportException
java.net.SocketTimeoutException: Read timed out
org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
at
org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:354)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_all_databases(ThriftHiveMetastore.java:625)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_all_databases(ThriftHiveMetastore.java:613)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:837)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
at
org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
at
org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
at
org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
at
org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
at
org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
at
org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
at
org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
at
org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
at
org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
at
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
... 47 more
2014-11-28 11:09:38,538 ERROR hive.log: Converting exception to
MetaException
2014-11-28 11:09:38,550 ERROR org.apache.tajo.catalog.CatalogServer:
org.apache.tajo.catalog.exception.CatalogException:
MetaException(message:Got exception:
org.apache.thrift.transport.TTransportException
java.net.SocketTimeoutException: Read timed out)
2014-11-28 11:09:38,550 ERROR
org.apache.tajo.catalog.AbstractCatalogClient:
com.google.protobuf.ServiceException:
org.apache.tajo.catalog.exception.CatalogException:
MetaException(message:Got exception:
org.apache.thrift.transport.TTransportException
java.net.SocketTimeoutException: Read timed out)
com.google.protobuf.ServiceException: com.google.protobuf.ServiceException:
org.apache.tajo.catalog.exception.CatalogException:
MetaException(message:Got exception:
org.apache.thrift.transport.TTransportException
java.net.SocketTimeoutException: Read timed out)
at
org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
at
org.apache.tajo.catalog.AbstractCatalogClient.existDatabase(AbstractCatalogClient.java:212)
at
org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.existDatabase(TajoMasterClientService.java:624)
at
org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:549)
at
org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
at
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.google.protobuf.ServiceException:
org.apache.tajo.catalog.exception.CatalogException:
MetaException(message:Got exception:
org.apache.thrift.transport.TTransportException
java.net.SocketTimeoutException: Read timed out)
at
org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:421)
at
org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:215)
at
org.apache.tajo.catalog.AbstractCatalogClient$9.call(AbstractCatalogClient.java:212)
at
org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
... 30 more
Caused by: org.apache.tajo.catalog.exception.CatalogException:
MetaException(message:Got exception:
org.apache.thrift.transport.TTransportException
java.net.SocketTimeoutException: Read timed out)
at
org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:372)
at
org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.existDatabase(CatalogServer.java:414)
... 33 more
Caused by: MetaException(message:Got exception:
org.apache.thrift.transport.TTransportException
java.net.SocketTimeoutException: Read timed out)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1102)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:839)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy12.getAllDatabases(Unknown Source)
at
org.apache.tajo.catalog.store.HCatalogStore.existDatabase(HCatalogStore.java:369)
... 34 more
ERROR: Exception was thrown. Caused by
org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
"default" does not exist
Exception in thread "main" com.google.protobuf.ServiceException:
org.apache.tajo.cli.InvalidClientSessionException: ERROR: database
"default" does not exist
at
org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:105)
at
org.apache.tajo.client.TajoClient.getCurrentDatabase(TajoClient.java:262)
at org.apache.tajo.cli.TajoCli.<init>(TajoCli.java:243)
at org.apache.tajo.cli.TajoCli.main(TajoCli.java:690)
Caused by: org.apache.tajo.cli.InvalidClientSessionException: ERROR:
database "default" does not exist
at
org.apache.tajo.client.TajoClient.checkSessionAndGet(TajoClient.java:249)
at org.apache.tajo.client.TajoClient.access$000(TajoClient.java:75)
at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:265)
at org.apache.tajo.client.TajoClient$1.call(TajoClient.java:262)
at
org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
... 3 more
-------------------------------------------------------------------------------------------------------------------------------------------------
On Fri, Nov 28, 2014 at 10:36 AM, Hyunsik Choi <hy...@apache.org> wrote:
> Hi Mai,
>
> Good to see you again!
>
> So far, we haven't performed QphH-based benchmark.
>
> Some of original TPC-H benchmark queries may not work in Tajo because
> scalar subquery, in-subquery, exist-subquery are not supported yet.
> So, you need to change slightly them into multiple queries.
>
> I haven't experienced such a situation you explained, could you share
> the error log that you met while you were executing multiple queries?
>
> Best regards,
> Hyunsik
>
>
>
> On Tue, Nov 25, 2014 at 12:12 PM, Thanh Mai <ma...@gmail.com> wrote:
> > Hi everyone,
> >
> > I want to find the performance results of Tajo with the TPC-H benchmark
> in
> > both power test and throughput test (and thus, a sample QphH value for
> > reference). However, from the slides in Tajo website, I found only the
> > response times of some queries.
> >
> > Have any of you tested Tajo in the TPC-H benchmark's throughput test or
> in
> > the multi-user situation ? If yes, could you share your experience ?
> >
> > When I test Tajo with the TPC-H benchmark's throughput test, some queries
> > failed occasionally. I use Hive catalog instead of Tajo catalog. It seems
> > that Tajo has a problem when getting information from Hive catalog
> > concurrently for multiple queries running in parallel. Have any of you
> > experienced a similar situation ? Could you share your experience and
> > insights ?
> >
> > Thank you very much!
> >
> > Sincerely,
> >
> > Mai Hai Thanh
>
Re: Tajo with TPC-H benchmark
Posted by Hyunsik Choi <hy...@apache.org>.
Hi Mai,
Good to see you again!
So far, we haven't performed QphH-based benchmark.
Some of original TPC-H benchmark queries may not work in Tajo because
scalar subquery, in-subquery, exist-subquery are not supported yet.
So, you need to change slightly them into multiple queries.
I haven't experienced such a situation you explained, could you share
the error log that you met while you were executing multiple queries?
Best regards,
Hyunsik
On Tue, Nov 25, 2014 at 12:12 PM, Thanh Mai <ma...@gmail.com> wrote:
> Hi everyone,
>
> I want to find the performance results of Tajo with the TPC-H benchmark in
> both power test and throughput test (and thus, a sample QphH value for
> reference). However, from the slides in Tajo website, I found only the
> response times of some queries.
>
> Have any of you tested Tajo in the TPC-H benchmark's throughput test or in
> the multi-user situation ? If yes, could you share your experience ?
>
> When I test Tajo with the TPC-H benchmark's throughput test, some queries
> failed occasionally. I use Hive catalog instead of Tajo catalog. It seems
> that Tajo has a problem when getting information from Hive catalog
> concurrently for multiple queries running in parallel. Have any of you
> experienced a similar situation ? Could you share your experience and
> insights ?
>
> Thank you very much!
>
> Sincerely,
>
> Mai Hai Thanh