You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mark Miller (JIRA)" <ji...@apache.org> on 2015/08/20 15:34:46 UTC
[jira] [Assigned] (SOLR-7948) MapReduceIndexerTool of solr 5.2.1
doesn't work with hadoop 2.7.1
[ https://issues.apache.org/jira/browse/SOLR-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller reassigned SOLR-7948:
---------------------------------
Assignee: Mark Miller
> MapReduceIndexerTool of solr 5.2.1 doesn't work with hadoop 2.7.1
> -----------------------------------------------------------------
>
> Key: SOLR-7948
> URL: https://issues.apache.org/jira/browse/SOLR-7948
> Project: Solr
> Issue Type: Bug
> Components: contrib - MapReduce
> Affects Versions: 5.2.1
> Environment: OS:suse 11
> JDK:java version "1.7.0_65"
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
> HADOOP:hadoop 2.7.1
> SOLR:5.2.1
> Reporter: davidchiu
> Assignee: Mark Miller
>
> When I used MapReduceIndexerTool of 5.2.1 to index files, I got follwoing errors,but I used 4.9.0's MapReduceIndexerTool, it did work with hadoop 2.7.1.
> Exception ERROR as following:
> INFO - 2015-08-20 11:44:45.155; [ ] org.apache.solr.hadoop.HeartBeater; Heart beat reporting class is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
> INFO - 2015-08-20 11:44:45.161; [ ] org.apache.solr.hadoop.SolrRecordWriter; Using this unpacked directory as solr home: /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip
> INFO - 2015-08-20 11:44:45.162; [ ] org.apache.solr.hadoop.SolrRecordWriter; Creating embedded Solr server with solrHomeDir: /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip, fs: DFS[DFSClient[clientName=DFSClient_attempt_1440040092614_0004_r_000001_0_1678264055_1, ugi=root (auth:SIMPLE)]], outputShardDir: hdfs://127.0.0.1:9000/tmp/outdir/reducers/_temporary/1/_temporary/attempt_1440040092614_0004_r_000001_0/part-r-00001
> INFO - 2015-08-20 11:44:45.194; [ ] org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/'
> INFO - 2015-08-20 11:44:45.206; [ ] org.apache.solr.hadoop.HeartBeater; HeartBeat thread running
> INFO - 2015-08-20 11:44:45.207; [ ] org.apache.solr.hadoop.HeartBeater; Issuing heart beat for 1 threads
> INFO - 2015-08-20 11:44:45.418; [ ] org.apache.solr.hadoop.SolrRecordWriter; Constructed instance information solr.home /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip (/usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip), instance dir /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/, conf dir /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/conf/, writing index to solr.data.dir hdfs://127.0.0.1:9000/tmp/outdir/reducers/_temporary/1/_temporary/attempt_1440040092614_0004_r_000001_0/part-r-00001/data, with permdir hdfs://127.0.0.10:9000/tmp/outdir/reducers/_temporary/1/_temporary/attempt_1440040092614_0004_r_000001_0/part-r-00001
> INFO - 2015-08-20 11:44:45.426; [ ] org.apache.solr.core.SolrXmlConfig; Loading container configuration from /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/solr.xml
> INFO - 2015-08-20 11:44:45.474; [ ] org.apache.solr.core.CorePropertiesLocator; Config-defined core root directory: /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip
> INFO - 2015-08-20 11:44:45.503; [ ] org.apache.solr.core.CoreContainer; New CoreContainer 1656436773
> INFO - 2015-08-20 11:44:45.503; [ ] org.apache.solr.core.CoreContainer; Loading cores into CoreContainer [instanceDir=/usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/]
> INFO - 2015-08-20 11:44:45.503; [ ] org.apache.solr.core.CoreContainer; loading shared library: /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/lib
> WARN - 2015-08-20 11:44:45.504; [ ] org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to add to classloader: lib (resolved as: /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/lib).
> INFO - 2015-08-20 11:44:45.520; [ ] org.apache.solr.handler.component.HttpShardHandlerFactory; created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
> FATAL - 2015-08-20 11:44:45.526; [ ] org.apache.hadoop.mapred.YarnChild; Error running child : java.lang.VerifyError: Bad return type
> Exception Details:
> Location:
> org/apache/solr/client/solrj/impl/HttpClientUtil.createClient(Lorg/apache/solr/common/params/SolrParams;Lorg/apache/http/conn/ClientConnectionManager;)Lorg/apache/http/impl/client/CloseableHttpClient; @62: areturn
> Reason:
> Type 'org/apache/http/impl/client/DefaultHttpClient' (current frame, stack[0]) is not assignable to 'org/apache/http/impl/client/CloseableHttpClient' (from method signature)
> Current Frame:
> bci: @62
> flags: { }
> locals: { 'org/apache/solr/common/params/SolrParams', 'org/apache/http/conn/ClientConnectionManager', 'org/apache/solr/common/params/ModifiableSolrParams', 'org/apache/http/impl/client/DefaultHttpClient' }
> stack: { 'org/apache/http/impl/client/DefaultHttpClient' }
> Bytecode:
> 0000000: bb00 0359 2ab7 0004 4db2 0005 b900 0601
> 0000010: 0099 001e b200 05bb 0007 59b7 0008 1209
> 0000020: b600 0a2c b600 0bb6 000c b900 0d02 00bb
> 0000030: 0011 592b b700 124e 2d2c b800 102d b0
> Stackmap Table:
> append_frame(@47,Object[#127])
> at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:166)
> at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:328)
> at org.apache.solr.hadoop.SolrRecordWriter.createEmbeddedSolrServer(SolrRecordWriter.java:163)
> at org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:119)
> at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:163)
> at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org