You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/02/27 18:54:00 UTC
[jira] [Resolved] (IMPALA-6593)
test_refresh_partition.py::TestRefreshPartition::()::test_add_data_and_refresh
during GVO
[ https://issues.apache.org/jira/browse/IMPALA-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-6593.
-----------------------------------
Resolution: Duplicate
Found in hive.log:
{noformat}
2018-02-27 08:21:10,181 WARN mapred.LocalJobRunner (LocalJobRunner.java:run(587)) - job_local939953830_0002
java.lang.Exception: java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:267)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:213)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:334)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:695)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:438)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:253)
... 12 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.initIOContext(HiveContextAwareRecordReader.java:171)
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.initIOContext(HiveContextAwareRecordReader.java:208)
at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:70)
... 17 more
{noformat}
> test_refresh_partition.py::TestRefreshPartition::()::test_add_data_and_refresh during GVO
> -----------------------------------------------------------------------------------------
>
> Key: IMPALA-6593
> URL: https://issues.apache.org/jira/browse/IMPALA-6593
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: Michael Ho
> Priority: Major
>
> test_refresh_partition.py failed with the following error during GVO.
> {noformat}
> 09:48:20 ] FAIL metadata/test_refresh_partition.py::TestRefreshPartition::()::test_add_data_and_refresh[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none]
> 09:48:20 ] =================================== FAILURES ===================================
> 09:48:20 ] TestRefreshPartition.test_add_data_and_refresh[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none]
> 09:48:20 ] [gw14] linux2 -- Python 2.7.12 /home/ubuntu/Impala/bin/../infra/python/env/bin/python
> 09:48:20 ] metadata/test_refresh_partition.py:103: in test_add_data_and_refresh
> 09:48:20 ] % table_name)
> 09:48:20 ] common/impala_test_suite.py:684: in run_stmt_in_hive
> 09:48:20 ] raise RuntimeError(stderr)
> 09:48:20 ] E RuntimeError: SLF4J: Class path contains multiple SLF4J bindings.
> 09:48:20 ] E SLF4J: Found binding in [jar:file:/home/ubuntu/Impala/toolchain/cdh_components/hbase-1.2.0-cdh5.15.0-SNAPSHOT/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 09:48:20 ] E SLF4J: Found binding in [jar:file:/home/ubuntu/Impala/toolchain/cdh_components/hadoop-2.6.0-cdh5.15.0-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 09:48:20 ] E SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> 09:48:20 ] E SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 09:48:20 ] E scan complete in 2ms
> 09:48:20 ] E Connecting to jdbc:hive2://localhost:11050
> 09:48:20 ] E Connected to: Apache Hive (version 1.1.0-cdh5.15.0-SNAPSHOT)
> 09:48:20 ] E Driver: Hive JDBC (version 1.1.0-cdh5.15.0-SNAPSHOT)
> 09:48:20 ] E Transaction isolation: TRANSACTION_REPEATABLE_READ
> 09:48:20 ] E INFO : Compiling command(queryId=ubuntu_20180227082121_16932ac7-79f6-42d9-a00a-09da622ee84b): insert into table test_add_data_and_refresh_e5ddf825.partition_test_table partition (y=333, z=5309) values (2)
> 09:48:20 ] E INFO : Semantic Analysis Completed
> 09:48:20 ] E INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_col0, type:int, comment:null)], properties:null)
> 09:48:20 ] E INFO : Completed compiling command(queryId=ubuntu_20180227082121_16932ac7-79f6-42d9-a00a-09da622ee84b); Time taken: 0.304 seconds
> 09:48:20 ] E INFO : Executing command(queryId=ubuntu_20180227082121_16932ac7-79f6-42d9-a00a-09da622ee84b): insert into table test_add_data_and_refresh_e5ddf825.partition_test_table partition (y=333, z=5309) values (2)
> 09:48:20 ] E INFO : Query ID = ubuntu_20180227082121_16932ac7-79f6-42d9-a00a-09da622ee84b
> 09:48:20 ] E INFO : Total jobs = 3
> 09:48:20 ] E INFO : Launching Job 1 out of 3
> 09:48:20 ] E INFO : Starting task [Stage-1:MAPRED] in serial mode
> 09:48:20 ] E INFO : Number of reduce tasks is set to 0 since there's no reduce operator
> 09:48:20 ] E INFO : number of splits:1
> 09:48:20 ] E INFO : Submitting tokens for job: job_local939953830_0002
> 09:48:20 ] E INFO : The url to track the job: http://localhost:8080/
> 09:48:20 ] E INFO : Job running in-process (local Hadoop)
> 09:48:20 ] E INFO : 2018-02-27 08:21:11,096 Stage-1 map = 0%, reduce = 0%
> 09:48:20 ] E ERROR : Ended Job = job_local939953830_0002 with errors
> 09:48:20 ] E ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> 09:48:20 ] E INFO : MapReduce Jobs Launched:
> 09:48:20 ] E INFO : Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL
> 09:48:20 ] E INFO : Total MapReduce CPU Time Spent: 0 msec
> 09:48:20 ] E INFO : Completed executing command(queryId=ubuntu_20180227082121_16932ac7-79f6-42d9-a00a-09da622ee84b); Time taken: 1.634 seconds
> 09:48:20 ] E Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
> 09:48:20 ] E Closing: 0: jdbc:hive2://localhost:11050
> 09:48:20 ] ---------------------------- Captured stderr setup -----------------------------
> 09:48:20 ] -- connecting to: localhost:21000
> 09:48:20 ] SET sync_ddl=False;
> 09:48:20 ] -- executing against localhost:21000
> 09:48:20 ] DROP DATABASE IF EXISTS `test_add_data_and_refresh_e5ddf825` CASCADE;
> 09:48:20 ]
> 09:48:20 ] SET sync_ddl=False;
> 09:48:20 ] -- executing against localhost:21000
> 09:48:20 ] CREATE DATABASE `test_add_data_and_refresh_e5ddf825`;
> 09:48:20 ]
> 09:48:20 ] MainThread: Created database "test_add_data_and_refresh_e5ddf825" for test ID "metadata/test_refresh_partition.py::TestRefreshPartition::()::test_add_data_and_refresh[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none]"
> 09:48:20 ] ----------------------------- Captured stderr call -----------------------------
> 09:48:20 ] -- executing against localhost:21000
> 09:48:20 ] create table test_add_data_and_refresh_e5ddf825.partition_test_table (x int) partitioned by (y int, z int);
> 09:48:20 ]
> 09:48:20 ] -- executing against localhost:21000
> 09:48:20 ] alter table test_add_data_and_refresh_e5ddf825.partition_test_table add partition (y=333, z=5309);
> 09:48:20 ]
> 09:48:20 ] -- executing against localhost:21000
> 09:48:20 ] select count(*) from test_add_data_and_refresh_e5ddf825.partition_test_table;
> 09:48:20 ]
> 09:48:20 ] ======= 1 failed, 1689 passed, 51 skipped, 36 xfailed in 2721.17 seconds =======
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)