You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by atp <an...@hcl.com> on 2014/06/19 13:10:51 UTC

Unable to start solr 4.8

Hi experts, 

i have cnfigured solrcloud, on three machines , zookeeper started with no
errors, tomcat log also no errors , solr log alos no errors reported but all
the tomcat configured solr clusterstate shows as 'down'



,8870931 [Thread-13] INFO  org.apache.solr.common.cloud.ZkStateReader  â
Updating cloud state from ZooKeeper...
8870934 [Thread-13] INFO  org.apache.solr.cloud.Overseer  â Update state
numShards=2 message={
  "operation":"state",
  "state":"down",
  "base_url":"http://10.***.***.28:7090/solr",
  "core":"collection1",
  "roles":null,
  "node_name":"10.***.***.28:7090_solr",
  "shard":"shard2",
  "collection":"collection1",
  "numShards":"2",
  "core_node_name":"10.***.***.28:7090_solr_collection1"}
8870939 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
8870942 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â A cluster state change: WatchedEvent state:SyncConnected
type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
(live nodes size: 5)
8919667 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â Updating live nodes... (4)
8933777 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â Updating live nodes... (3)
8965906 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â Updating live nodes... (4)
8965994 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
8965997 [Thread-13] INFO  org.apache.solr.common.cloud.ZkStateReader  â
Updating cloud state from ZooKeeper...
8966000 [Thread-13] INFO  org.apache.solr.cloud.Overseer  â Update state
numShards=2 message={
  "operation":"state",
  "state":"down",
  "base_url":"http://10.***.***.29:7070/solr",
  "core":"collection1",
  "roles":null,
  "node_name":"10.***.***.29:7070_solr",
  "shard":"shard1",
  "collection":"collection1",
  "numShards":"2",
  "core_node_name":"110.***.***.29:7070_solr_collection1"}
8966006 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
8966008 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â A cluster state change: WatchedEvent state:SyncConnected
type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
(live nodes size: 4)
8986466 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â Updating live nodes... (5)
8986648 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
8986652 [Thread-13] INFO  org.apache.solr.common.cloud.ZkStateReader  â
Updating cloud state from ZooKeeper...
8986654 [Thread-13] INFO  org.apache.solr.cloud.Overseer  â Update state
numShards=2 message={
  "operation":"state",
  "state":"down",
  "base_url":"http://10.***.***.30:7080/solr",
  "core":"collection1",
  "roles":null,
  "node_name":"10.***.***.30:7080_solr",
  "shard":"shard1",
  "collection":"collection1",
  "numShards":"2",
  "core_node_name":"10.***.***.30:7080_solr_collection1"}
8986661 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
8986666 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â A cluster state change: WatchedEvent state:SyncConnected
type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
(live nodes size: 5)
9008407 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â Updating live nodes... (6)



when i browse the 28,29 and 30th solr url , its throwing error like, 


HTTP Status 500 - {msg=SolrCore 'collection1' is not available due to init
failure: Index locked for write for core
collection1,trace=org.apache.solr.common.SolrException: SolrCore
'collection1' is not available due to init failure: Index locked for write
for core collection1 at
org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:753) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745) Caused by:
org.apache.solr.common.SolrException: Index locked for write for core
collection1 at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) at
org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) at
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at
java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at
java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more Caused by: org.apache.lucene.store.LockObtainFailedException:
Index locked for write for core collection1 at
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:503) at
org.apache.solr.core.SolrCore.<init>(SolrCore.java:772) ... 10 more
,code=500}

Please help on this to resolve? 

Thanks in Advance 
ATP






--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-start-solr-4-8-tp4142810.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unable to start solr 4.8

Posted by atp <an...@hcl.com>.
Thank you so much Markus , 

I have removed the contents from Index , now its working but one of the node
went Recovering state, the log says,  please help this to make to live. 

"â Unable to get file names for indexCommit generation: 2"



12118803 [qtp1490747277-11] INFO  org.apache.solr.core.SolrCore  â
[collection1] webapp=/solr path=/replication
params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2}
status=0 QTime=3
12182820 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
12182824 [Thread-13] INFO  org.apache.solr.common.cloud.ZkStateReader  â
Updating cloud state from ZooKeeper...
12182827 [qtp1490747277-12] INFO 
org.apache.solr.handler.admin.CoreAdminHandler  â Going to wait for
coreNodeName: 10.137.12.247:7080_solr_collection1, state: recovering,
checkLive: true, onlyIfLeader: true
12182828 [Thread-13] INFO  org.apache.solr.cloud.Overseer  â Update state
numShards=2 message={
  "operation":"state",
  "state":"recovering",
  "base_url":"http://10.***.***.29:7080/solr",
  "core":"collection1",
  "roles":null,
  "node_name":"10.***.***.29:7080_solr",
  "shard":"shard1",
  "collection":"collection1",
  "numShards":"2",
  "core_node_name":"10.***.***.29:7080_solr_collection1"}
12182834 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
12182839 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â A cluster state change: WatchedEvent state:SyncConnected
type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
(live nodes size: 6)
12182853 [qtp1490747277-12] INFO  org.apache.solr.common.cloud.ZkStateReader 
â Updating cloud state from ZooKeeper...
12182856 [qtp1490747277-12] INFO 
org.apache.solr.handler.admin.CoreAdminHandler  â Will wait a max of 183
seconds to see collection1 (shard1 of collection1) have state: recovering
12182856 [qtp1490747277-12] INFO 
org.apache.solr.handler.admin.CoreAdminHandler  â Waited coreNodeName:
10.137.12.247:7080_solr_collection1, state: recovering, checkLive: true,
onlyIfLeader: true for: 0 seconds.
12182858 [qtp1490747277-12] INFO  org.apache.solr.servlet.SolrDispatchFilter 
â [admin] webapp=null path=/admin/cores
params={coreNodeName=10.137.12.247:7080_solr_collection1&onlyIfLeaderActive=true&state=recovering&nodeName=10.137.12.247:7080_solr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2}
status=0 QTime=31
12184865 [qtp1490747277-19] INFO  org.apache.solr.update.UpdateHandler  â
start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
12184865 [qtp1490747277-19] INFO  org.apache.solr.update.UpdateHandler  â No
uncommitted changes. Skipping IW.commit.
12184867 [qtp1490747277-19] INFO  org.apache.solr.update.UpdateHandler  â
end_commit_flush
12184867 [qtp1490747277-19] INFO 
org.apache.solr.update.processor.LogUpdateProcessor  â [collection1]
webapp=/solr path=/update
params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false}
{commit=} 0 3
12184873 [qtp1490747277-18] INFO  org.apache.solr.core.SolrCore  â
[collection1] webapp=/solr path=/replication
params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0
QTime=1
12184878 [qtp1490747277-18] ERROR org.apache.solr.handler.ReplicationHandler 
â Unable to get file names for indexCommit generation: 2
java.io.FileNotFoundException: _0.fnm
        at
org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:260)
        at
org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:177)
        at
org.apache.solr.handler.ReplicationHandler.getFileList(ReplicationHandler.java:421)
        at
org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:209)
        at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
        at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)

Thanks ,
ATP




--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-start-solr-4-8-tp4142810p4142816.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unable to start solr 4.8

Posted by Erick Erickson <er...@gmail.com>.
Try nuking the entire data directory. As in rm -rf .../data. Although
why it should report a problem with the lock file I'm not quite sure.

Best,
Erick

On Mon, Jun 23, 2014 at 12:10 AM, atp <an...@hcl.com> wrote:
> hi ,
>
> After rebooting all the cluster machine , we unable to start the solr .
> getting error messge like in solr log file,
>
> We removed index directory contents solr/<collection_name>/data/index.*/
> and restared but getting 'â Recovery failed' from one of the node.  please
> help to resolve this.
>
> 666058 [RecoveryThread] ERROR org.apache.solr.cloud.RecoveryStrategy  â
> Error while trying to recover.
> core=collection1:java.util.concurrent.ExecutionException:
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
> SolrCore 'collection1' is not available due to init failure: Index locked
> for write for core collection1
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:188)
>         at
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:611)
>         at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:370)
>         at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)
> Caused by:
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
> SolrCore 'collection1' is not available due to init failure: Index locked
> for write for core collection1
>         at
> org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:245)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:241)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
>
> 666058 [RecoveryThread] ERROR org.apache.solr.cloud.RecoveryStrategy  â
> Recovery failed - trying again... (8) core=collection1
> 666058 [RecoveryThread] INFO  org.apache.solr.cloud.RecoveryStrategy  â Wait
> 512.0 seconds before trying to recover again (9)
> 666066 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â A cluster state change: WatchedEvent state:SyncConnected
> type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
> (live nodes size: 3)
>
>
> While accessing through Web url , getting
>
>
> HTTP Status 500 - {msg=SolrCore 'collection1' is not available due to init
> failure: Index locked for write for core
> collection1,trace=org.apache.solr.common.SolrException: SolrCore
> 'collection1' is not available due to init failure: Index locked for write
> for core collection1 at
> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:753) at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745) Caused by:
> org.apache.solr.common.SolrException: Index locked for write for core
> collection1 at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) at
> org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) at
> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at
> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at
> java.util.concurrent.FutureTask.run(FutureTask.java:262) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at
> java.util.concurrent.FutureTask.run(FutureTask.java:262) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more Caused by: org.apache.lucene.store.LockObtainFailedException:
> Index locked for write for core collection1 at
> org.apache.solr.core.SolrCore.initIndex(SolrCore.java:503) at
> org.apache.solr.core.SolrCore.<init>(SolrCore.java:772) ... 10 more
> ,code=500}
>
> We dont have any indexed data on this , please guide us, is there anything
> needs to be removed to bring up this?
>
>
> Thanks,
> ATP
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-start-solr-4-8-tp4142810p4143370.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unable to start solr 4.8

Posted by atp <an...@hcl.com>.
hi ,

After rebooting all the cluster machine , we unable to start the solr .
getting error messge like in solr log file, 

We removed index directory contents solr/<collection_name>/data/index.*/ 
and restared but getting 'â Recovery failed' from one of the node.  please
help to resolve this. 

666058 [RecoveryThread] ERROR org.apache.solr.cloud.RecoveryStrategy  â
Error while trying to recover.
core=collection1:java.util.concurrent.ExecutionException:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
SolrCore 'collection1' is not available due to init failure: Index locked
for write for core collection1
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:188)
        at
org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:611)
        at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:370)
        at
org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)
Caused by:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
SolrCore 'collection1' is not available due to init failure: Index locked
for write for core collection1
        at
org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
        at
org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:245)
        at
org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:241)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

666058 [RecoveryThread] ERROR org.apache.solr.cloud.RecoveryStrategy  â
Recovery failed - trying again... (8) core=collection1
666058 [RecoveryThread] INFO  org.apache.solr.cloud.RecoveryStrategy  â Wait
512.0 seconds before trying to recover again (9)
666066 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader 
â A cluster state change: WatchedEvent state:SyncConnected
type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
(live nodes size: 3)


While accessing through Web url , getting 


HTTP Status 500 - {msg=SolrCore 'collection1' is not available due to init
failure: Index locked for write for core
collection1,trace=org.apache.solr.common.SolrException: SolrCore
'collection1' is not available due to init failure: Index locked for write
for core collection1 at
org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:753) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745) Caused by:
org.apache.solr.common.SolrException: Index locked for write for core
collection1 at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) at
org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) at
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at
java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at
java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more Caused by: org.apache.lucene.store.LockObtainFailedException:
Index locked for write for core collection1 at
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:503) at
org.apache.solr.core.SolrCore.<init>(SolrCore.java:772) ... 10 more
,code=500}

We dont have any indexed data on this , please guide us, is there anything
needs to be removed to bring up this?


Thanks,
ATP




--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-start-solr-4-8-tp4142810p4143370.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unable to start solr 4.8

Posted by atp <an...@hcl.com>.
Hi Markus ,

It been recoverd automatically after several attempts ,

once again thanks a lot for your help. 

Now all the nodes are became live. 


[zk: Hadoop-Main:7001(CONNECTED) 0] get /clusterstate.json
{"collection1":{
    "shards":{
      "shard1":{
        "range":"80000000-ffffffff",
        "state":"active",
        "replicas":{
          "core_node1":{
            "state":"active",
            "base_url":"http://10.***.***.28:8983/solr",
            "core":"collection1",
            "node_name":"10.***.***.28:8983_solr",
            "leader":"true"},
          "core_node3":{
            "state":"active",
            "base_url":"http://10.***.***.30:8983/solr",
            "core":"collection1",
            "node_name":"10.***.***.30:8983_solr"},
          "10.***.***.28:7070_solr_collection1":{
            "state":"active",
            "base_url":"http://10.***.***.28:7070/solr",
            "core":"collection1",
            "node_name":"10.***.***.28:7070_solr"},
          "10.***.***.29:7080_solr_collection1":{
            "state":"active",
            "base_url":"http://10.***.***.29:7080/solr",
            "core":"collection1",
            "node_name":"10.***.***.29:7080_solr"}}},
      "shard2":{
        "range":"0-7fffffff",
        "state":"active",
        "replicas":{
          "core_node2":{
            "state":"active",
            "base_url":"http://10.***.***.29:8983/solr",
            "core":"collection1",
            "node_name":"10.***.***.29:8983_solr",
            "leader":"true"},
          "10.***.***.30:7090_solr_collection1":{
            "state":"active",
            "base_url":"http://10.***.***.30:7090/solr",
            "core":"collection1",
            "node_name":"10.***.***.30:7090_solr"}}}},
    "maxShardsPerNode":"1",
    "router":{"name":"compositeId"},
    "replicationFactor":"1",
    "autoCreated":"true"}}



Regards,
ATP.



--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-start-solr-4-8-tp4142810p4142822.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unable to start solr 4.8

Posted by Markus Jelsma <ma...@openindex.io>.
Hi - remove the lock file in your solr/<collection_name>/data/index.*/ 
directory.

Markus

On Thursday, June 19, 2014 04:10:51 AM atp wrote:
> Hi experts,
> 
> i have cnfigured solrcloud, on three machines , zookeeper started with no
> errors, tomcat log also no errors , solr log alos no errors reported but all
> the tomcat configured solr clusterstate shows as 'down'
> 
> 
> 
> ,8870931 [Thread-13] INFO  org.apache.solr.common.cloud.ZkStateReader  â
> Updating cloud state from ZooKeeper...
> 8870934 [Thread-13] INFO  org.apache.solr.cloud.Overseer  â Update state
> numShards=2 message={
>   "operation":"state",
>   "state":"down",
>   "base_url":"http://10.***.***.28:7090/solr",
>   "core":"collection1",
>   "roles":null,
>   "node_name":"10.***.***.28:7090_solr",
>   "shard":"shard2",
>   "collection":"collection1",
>   "numShards":"2",
>   "core_node_name":"10.***.***.28:7090_solr_collection1"}
> 8870939 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 8870942 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â A cluster state change: WatchedEvent state:SyncConnected
> type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
> (live nodes size: 5)
> 8919667 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â Updating live nodes... (4)
> 8933777 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â Updating live nodes... (3)
> 8965906 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â Updating live nodes... (4)
> 8965994 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 8965997 [Thread-13] INFO  org.apache.solr.common.cloud.ZkStateReader  â
> Updating cloud state from ZooKeeper...
> 8966000 [Thread-13] INFO  org.apache.solr.cloud.Overseer  â Update state
> numShards=2 message={
>   "operation":"state",
>   "state":"down",
>   "base_url":"http://10.***.***.29:7070/solr",
>   "core":"collection1",
>   "roles":null,
>   "node_name":"10.***.***.29:7070_solr",
>   "shard":"shard1",
>   "collection":"collection1",
>   "numShards":"2",
>   "core_node_name":"110.***.***.29:7070_solr_collection1"}
> 8966006 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 8966008 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â A cluster state change: WatchedEvent state:SyncConnected
> type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
> (live nodes size: 4)
> 8986466 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â Updating live nodes... (5)
> 8986648 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 8986652 [Thread-13] INFO  org.apache.solr.common.cloud.ZkStateReader  â
> Updating cloud state from ZooKeeper...
> 8986654 [Thread-13] INFO  org.apache.solr.cloud.Overseer  â Update state
> numShards=2 message={
>   "operation":"state",
>   "state":"down",
>   "base_url":"http://10.***.***.30:7080/solr",
>   "core":"collection1",
>   "roles":null,
>   "node_name":"10.***.***.30:7080_solr",
>   "shard":"shard1",
>   "collection":"collection1",
>   "numShards":"2",
>   "core_node_name":"10.***.***.30:7080_solr_collection1"}
> 8986661 [main-EventThread] INFO  org.apache.solr.cloud.DistributedQueue  â
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 8986666 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â A cluster state change: WatchedEvent state:SyncConnected
> type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
> (live nodes size: 5)
> 9008407 [main-EventThread] INFO  org.apache.solr.common.cloud.ZkStateReader
> â Updating live nodes... (6)
> 
> 
> 
> when i browse the 28,29 and 30th solr url , its throwing error like,
> 
> 
> HTTP Status 500 - {msg=SolrCore 'collection1' is not available due to init
> failure: Index locked for write for core
> collection1,trace=org.apache.solr.common.SolrException: SolrCore
> 'collection1' is not available due to init failure: Index locked for write
> for core collection1 at
> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:753) at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
> 347) at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
> 207) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:241) at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:208) at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:220) at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:122) at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171
> ) at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
> ) at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :116) at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Proce
> ssor.java:1040) at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Abstrac
> tProtocol.java:607) at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:
> 316) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
> 45) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
> 15) at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.ja
> va:61) at java.lang.Thread.run(Thread.java:745) Caused by:
> org.apache.solr.common.SolrException: Index locked for write for core
> collection1 at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) at
> org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) at
> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at
> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at
> java.util.concurrent.FutureTask.run(FutureTask.java:262) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at
> java.util.concurrent.FutureTask.run(FutureTask.java:262) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
> 45) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
> 15) ... 1 more Caused by: org.apache.lucene.store.LockObtainFailedException:
> Index locked for write for core collection1 at
> org.apache.solr.core.SolrCore.initIndex(SolrCore.java:503) at
> org.apache.solr.core.SolrCore.<init>(SolrCore.java:772) ... 10 more
> ,code=500}
> 
> Please help on this to resolve?
> 
> Thanks in Advance
> ATP
> 
> 
> 
> 
> 
> 
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Unable-to-start-solr-4-8-tp4142810.html
> Sent from the Solr - User mailing list archive at Nabble.com.