You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "Hyunsik Choi (JIRA)" <ji...@apache.org> on 2014/03/10 15:49:44 UTC
[jira] [Resolved] (TAJO-655) QueryMaster sent "Select query"
command to TajoWorker,but the TajoWorker don't working
[ https://issues.apache.org/jira/browse/TAJO-655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi resolved TAJO-655.
-------------------------------
Resolution: Not A Problem
> QueryMaster sent "Select query" command to TajoWorker,but the TajoWorker don't working
> ---------------------------------------------------------------------------------------
>
> Key: TAJO-655
> URL: https://issues.apache.org/jira/browse/TAJO-655
> Project: Tajo
> Issue Type: Bug
> Components: query master, worker
> Affects Versions: 0.8-incubating
> Environment: CentOS 6.3 32bit,jdk 1.7,hadoop 2.2.0
> Reporter: Andrew Guo
> Fix For: 0.8-incubating
>
>
> Hi
> Yesterday ,You provide tajo branch-0.8.0 already installed. In the http://xxx:26080 web ui,the TajaWorker is live .In the TSQL interface input "select * from test" command,but Prompt "Internal error!" error.
> TajoWorker error log content:
> =======================================================
> Block Id: eb_1393922377833_0005_000001 [ROOT]
> =======================================================
> SCAN(0) on test
> => target list: test.id (INT4), test.interface (TEXT), test.start_time (TEXT), test.byte (TEXT)
> => out schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time (TEXT),test.byte (TEXT)}
> => in schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time (TEXT),test.byte (TEXT)}
> =======================================================
> Block Id: eb_1393922377833_0005_000002 [TERMINAL]
> =======================================================
> 2014-03-05 10:48:18,219 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:start(101)) - AsyncDispatcher started:q_1393922377833_0005
> 2014-03-05 10:48:18,222 INFO querymaster.Query (Query.java:handle(641)) - Processing q_1393922377833_0005 of type START
> 2014-03-05 10:48:18,223 INFO querymaster.SubQuery (SubQuery.java:initTaskScheduler(661)) - org.apache.tajo.master.DefaultTaskScheduler is chosen for the task scheduling
> 2014-03-05 10:48:18,230 INFO storage.AbstractStorageManager (AbstractStorageManager.java:listStatus(382)) - Total input paths to process : 1
> 2014-03-05 10:48:18,236 ERROR querymaster.SubQuery (SubQuery.java:transition(645)) - SubQuery (eb_1393922377833_0005_000001) ERROR:
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
> at org.apache.tajo.storage.AbstractStorageManager.newScannerInstance(AbstractStorageManager.java:661)
> at org.apache.tajo.storage.StorageManager.getScanner(StorageManager.java:60)
> at org.apache.tajo.storage.AbstractStorageManager.getScanner(AbstractStorageManager.java:106)
> at org.apache.tajo.storage.AbstractStorageManager.getFileScanner(AbstractStorageManager.java:94)
> at org.apache.tajo.storage.AbstractStorageManager.isSplittable(AbstractStorageManager.java:399)
> at org.apache.tajo.storage.AbstractStorageManager.getSplits(AbstractStorageManager.java:554)
> at org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.scheduleFragmentsForLeafQuery(SubQuery.java:872)
> at org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.schedule(SubQuery.java:778)
> at org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:629)
> at org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:610)
> at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tajo.master.querymaster.SubQuery.handle(SubQuery.java:587)
> at org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:332)
> at org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:321)
> at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tajo.master.querymaster.Query.handle(Query.java:646)
> at org.apache.tajo.master.querymaster.Query.handle(Query.java:58)
> at org.apache.tajo.master.TajoAsyncDispatcher.dispatch(TajoAsyncDispatcher.java:137)
> at org.apache.tajo.master.TajoAsyncDispatcher$1.run(TajoAsyncDispatcher.java:79)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.tajo.storage.AbstractStorageManager.newScannerInstance(AbstractStorageManager.java:659)
> ... 25 more
> Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
> at org.apache.tajo.storage.CSVFile$CSVScanner.<init>(CSVFile.java:245)
> ... 30 more
> Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
> ... 32 more
> 2014-03-05 10:48:18,237 INFO querymaster.Query (Query.java:handle(654)) - q_1393922377833_0005 Query Transitioned from QUERY_NEW to QUERY_RUNNING
> 2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(641)) - Processing q_1393922377833_0005 of type SUBQUERY_COMPLETED
> 2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(641)) - Processing q_1393922377833_0005 of type QUERY_COMPLETED
> 2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(654)) - q_1393922377833_0005 Query Transitioned from QUERY_RUNNING to QUERY_ERROR
> 2014-03-05 10:48:18,238 INFO querymaster.QueryMasterTask (QueryMasterTask.java:handle(266)) - Query completion notified from q_1393922377833_0005
> 2014-03-05 10:48:18,238 INFO querymaster.QueryMasterTask (QueryMasterTask.java:handle(277)) - Query final state: QUERY_ERROR
> 2014-03-05 10:48:18,239 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(172)) - Stopping QueryMasterTask:q_1393922377833_0005
> 2014-03-05 10:48:18,246 WARN master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:stop(115)) - Interrupted Exception while stopping
> 2014-03-05 10:48:18,247 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher stopped:q_1393922377833_0005
> 2014-03-05 10:48:18,248 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(200)) - Stopped QueryMasterTask:q_1393922377833_0005
> 2014-03-05 10:48:18,248 INFO querymaster.QueryMaster (QueryMaster.java:cleanup(167)) - cleanup query resources : q_1393922377833_0005
> 2014-03-05 10:48:18,419 INFO worker.TajoWorkerClientService (TajoWorkerClientService.java:closeQuery(211)) - Stop Query:q_1393922377833_0005
> 2014-03-05 10:48:18,844 INFO querymaster.QueryMasterManagerService (QueryMasterManagerService.java:executeQuery(224)) - Receive executeQuery request:q_1393922377833_0006
> 2014-03-05 10:48:18,845 INFO querymaster.QueryMaster (QueryMaster.java:handle(374)) - Start QueryStartEventHandler:q_1393922377833_0006
> 2014-03-05 10:48:18,853 INFO querymaster.QueryMasterTask (QueryMasterTask.java:initStagingDir(403)) - The staging dir 'null' is created.
> 2014-03-05 10:48:18,943 INFO rewrite.BasicQueryRewriteEngine (BasicQueryRewriteEngine.java:rewrite(64)) - The rule "ProjectionPushDown " rewrites the query.
> 2014-03-05 10:48:18,944 INFO global.GlobalPlanner (GlobalPlanner.java:build(103)) -
> -----------------------------
> Query Block Graph
> The QueryMaster log content:
> -----------------------------
> Query Block Graph
> -----------------------------
> |-#ROOT
> -----------------------------
> Optimization Log:
> -----------------------------
> SCAN(0) on test
> => target list: test.id (INT4), test.interface (TEXT), test.start_time (TEXT), test.byte (TEXT)
> => out schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time (TEXT),test.byte (TEXT)}
> => in schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time (TEXT),test.byte (TEXT)}
> 2014-03-05 10:48:11,760 INFO master.GlobalEngine (GlobalEngine.java:createLogicalPlan(252)) - =============================================
> 2014-03-05 10:48:11,761 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:start(101)) - AsyncDispatcher started:QueryInProgress:q_1393922377833_0003
> 2014-03-05 10:48:11,761 INFO querymaster.QueryInProgress (QueryInProgress.java:startQueryMaster(151)) - Initializing QueryInProgress for QueryID=q_1393922377833_0003
> 2014-03-05 10:48:11,761 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:allocateQueryMaster(201)) - q_1393922377833_0003's QueryMaster is host:slave2, port=28093,28091,28092,53265, slots=m:1024,d:1.0,c:1, used=m:512,d:0.0,c:0
> 2014-03-05 10:48:11,762 INFO querymaster.QueryInProgress (QueryInProgress.java:connectQueryMaster(194)) - Connect to QueryMaster:slave2/192.168.1.79:28093
> 2014-03-05 10:48:11,767 INFO querymaster.QueryInProgress (QueryInProgress.java:submmitQueryToMaster(215)) - Call executeQuery to :slave2:28093,q_1393922377833_0003
> 2014-03-05 10:48:12,013 INFO querymaster.QueryInProgress (QueryInProgress.java:heartbeat(246)) - Received QueryMaster heartbeat:q_1393922377833_0003state=QUERY_RUNNING,progress=NaN, queryMaster=host:slave2, port=28093,28091,28092,0, slots=m:0,d:0.0,c:0, used=m:0,d:0.0,c:0
> 2014-03-05 10:48:12,015 INFO querymaster.QueryInProgress (QueryInProgress.java:heartbeat(246)) - Received QueryMaster heartbeat:q_1393922377833_0003state=QUERY_ERROR,progress=NaN, queryMaster=host:slave2, port=28093,28091,28092,0, slots=m:0,d:0.0,c:0, used=m:0,d:0.0,c:0
> 2014-03-05 10:48:12,015 INFO querymaster.QueryJobManager (QueryJobManager.java:stopQuery(139)) - Stop QueryInProgress:q_1393922377833_0003
> 2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress (QueryInProgress.java:stop(104)) - =========================================================
> 2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress (QueryInProgress.java:stop(105)) - Stop query:q_1393922377833_0003
> 2014-03-05 10:48:12,016 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:stopQueryMaster(634)) - release QueryMaster resource:q_1393922377833_0003,host:slave2, port=28093,28091,28092,53265, slots=m:1024,d:1.0,c:1, used=m:0,d:0.0,c:0
> 2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress (QueryInProgress.java:stop(113)) - q_1393922377833_0003 QueryMaster stopped
> 2014-03-05 10:48:12,019 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher stopped:QueryInProgress:q_1393922377833_0003
> 2014-03-05 10:48:16,253 INFO master.GlobalEngine (GlobalEngine.java:executeQuery(107)) - SQL: select * from test
> 2014-03-05 10:48:16,253 INFO master.GlobalEngine (GlobalEngine.java:executeQuery(126)) - hive.query.mode:false
> 2014-03-05 10:48:16,342 INFO master.GlobalEngine (GlobalEngine.java:createLogicalPlan(246)) - =============================================
> 2014-03-05 10:48:16,342 INFO master.GlobalEngine (GlobalEngine.java:createLogicalPlan(247)) - Non Optimized Query:
> -----------------------------
> Query Block Graph
> -----------------------------
--
This message was sent by Atlassian JIRA
(v6.2#6252)