You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by christopher palm <cp...@gmail.com> on 2014/05/21 20:08:27 UTC
SOLR 4.8 Collections API Create Collection Fails on Tomcat
Hello,
I am trying to use the collections API to add in a secondary collection
similar to what is described in this article
http://heliosearch.org/solrcloud-assigning-nodes-machines/
It works fine on a jetty setup where bootstrap config dir is specified.
When I try to run this on a tomcat solr that has the example configs
already in zookeeper from an upconfig it fails
http://SERVERADDRESS:8080/solr/admin/collections?action=CREATE&name=realtimecollection&collection.configName=collection1&numShards=1&createNodeSet=SERVERADDRESS:8080_solr
": Unable to create core: realtimecollection_shard1_replica1 Caused by:
null"
Any ideas why this fails?
>From the logs it seems to be trying to create this in the same folder as
the solr 4.8 war.
Below are the logs I see and the startup config arguments I am passing into
tomcat/SOLR.
Thanks,
Chris
export SOLR_HOME=/ebs-data/bin/solr
export JAVA_OPTS="$JAVA_OPTS -DzkHost=ZOOKEEPERCLUSTERADDRESS/app/test"
export JAVA_OPTS="$JAVA_OPTS -DnumShards=2"
export JAVA_OPTS="$JAVA_OPTS -Dcollection.configName=collection1 "
export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/ebs-data/solr/conf "
export JAVA_OPTS="$JAVA_OPTS -Dsolr.data.dir=/ebs-data/solr/data "
– A cluster state change: WatchedEvent state:SyncConnected
type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
(live nodes size: 1)
176300 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
INFO org.apache.solr.cloud.OverseerCollectionProcessor – Creating
SolrCores for new collection, shardNames [shard1] , replicationFactor : 1
176300 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
INFO org.apache.solr.cloud.OverseerCollectionProcessor – Creating shard
realtimecollection_shard1_replica1 as part of slice shard1 of collection
realtimecollection on SERVERADDRESS:8080_solr
178443 [http-bio-8080-exec-13] INFO
org.apache.solr.handler.admin.CoreAdminHandler – core create command
numShards=1&shard=shard1&collection.configName=collection1&name=realtimecollection_shard1_replica1&action=CREATE&collection=realtimecollection&wt=javabin&qt=/admin/cores&version=2
178540 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
publishing core=realtimecollection_shard1_replica1 state=down
collection=realtimecollection
178568 [localhost-startStop-1-EventThread] INFO
org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
/overseer/queue state: SyncConnected type NodeChildrenChanged
178569 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
waiting to find shard id in clusterstate for
realtimecollection_shard1_replica1
178641 [Thread-2] INFO org.apache.solr.common.cloud.ZkStateReader –
Updating cloud state from ZooKeeper...
178667 [Thread-2] INFO org.apache.solr.cloud.Overseer – Update state
numShards=1 message={
"operation":"state",
"state":"down",
"base_url":"http://SERVERADDRESS:8080/solr",
"core":"realtimecollection_shard1_replica1",
"roles":null,
"node_name":"SERVERADDRESS:8080_solr",
"shard":"shard1",
"collection":"realtimecollection",
"numShards":"1",
"core_node_name":"SERVERADDRESS:8080_solr_realtimecollection_shard1_replica1"}
178771 [localhost-startStop-1-EventThread] INFO
org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
/overseer/queue state: SyncConnected type NodeChildrenChanged
178798 [localhost-startStop-1-EventThread] INFO
org.apache.solr.common.cloud.ZkStateReader – A cluster state change:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
179572 [http-bio-8080-exec-13] INFO
org.apache.solr.core.SolrXMLCoresLocator – Persisted core descriptions to
/etc/solr/solr.xml
179573 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
Check for collection zkNode:realtimecollection
179600 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
Collection zkNode exists
179600 [http-bio-8080-exec-13] INFO
org.apache.solr.common.cloud.ZkStateReader – Load collection config
from:/collections/realtimecollection
179654 [http-bio-8080-exec-13] INFO
org.apache.solr.common.cloud.ZkStateReader –
path=/collections/realtimecollection configName=collection1 specified
config exists in ZooKeeper
179655 [http-bio-8080-exec-13] INFO
org.apache.solr.core.SolrResourceLoader – new SolrResourceLoader for
directory: '/etc/solr/realtimecollection_shard1_replica1/'
179900 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrConfig –
Adding specified lib dirs to ClassLoader
179901 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../contrib/extraction/lib (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../contrib/extraction/lib).
179902 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../dist/ (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../dist).
179902 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../contrib/clustering/lib/ (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../contrib/clustering/lib).
179903 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../dist/ (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../dist).
179903 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../contrib/langid/lib/ (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../contrib/langid/lib).
179904 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../dist/ (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../dist).
179904 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../contrib/velocity/lib (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../contrib/velocity/lib).
179904 [http-bio-8080-exec-13] WARN
org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
to add to classloader: ../../../dist/ (resolved as:
/etc/solr/realtimecollection_shard1_replica1/../../../dist).
180011 [http-bio-8080-exec-13] INFO org.apache.solr.update.SolrIndexConfig
– IndexWriter infoStream solr logging is enabled
180038 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrConfig –
Using Lucene MatchVersion: LUCENE_48
180185 [http-bio-8080-exec-13] INFO org.apache.solr.core.Config – Loaded
SolrConfig: solrconfig.xml
180285 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema –
Reading Solr Schema from schema.xml
180524 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema –
[realtimecollection_shard1_replica1] Schema name=example
182144 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema –
unique key field: id
182147 [http-bio-8080-exec-13] INFO org.apache.solr.rest.RestManager –
Registered ManagedResource impl
org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path
/schema/analysis/stopwords/english
182148 [http-bio-8080-exec-13] INFO org.apache.solr.rest.RestManager –
Registered ManagedResource impl
org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager
for path /schema/analysis/synonyms/english
183114 [http-bio-8080-exec-13] INFO
org.apache.solr.schema.FileExchangeRateProvider – Reloading exchange
rates from file currency.xml
183175 [http-bio-8080-exec-13] INFO
org.apache.solr.schema.FileExchangeRateProvider – Reloading exchange
rates from file currency.xml
185157 [http-bio-8080-exec-13] INFO org.apache.solr.core.CoreContainer –
Creating SolrCore 'realtimecollection_shard1_replica1' using configuration
from collection realtimecollection
185158 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
solr.NRTCachingDirectoryFactory
185158 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
[realtimecollection_shard1_replica1] Opening new SolrCore at
/etc/solr/realtimecollection_shard1_replica1/, dataDir=/ebs-data/solr/data/
185159 [http-bio-8080-exec-13] INFO org.apache.solr.core.JmxMonitoredMap
– JMX monitoring is enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@1de10458
185160 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
[realtimecollection_shard1_replica1] Added SolrEventListener for
newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
185160 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
[realtimecollection_shard1_replica1] Added SolrEventListener for
firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static
firstSearcher warming in solrconfig.xml}]}
185161 [http-bio-8080-exec-13] INFO
org.apache.solr.core.CachingDirectoryFactory – return new directory for
/ebs-data/solr/data
185162 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – New
index directory detected: old=null new=/ebs-data/solr/data/index/
185163 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
created json: solr.JSONResponseWriter
185163 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
adding lazy queryResponseWriter: solr.VelocityResponseWriter
185164 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
created velocity: solr.VelocityResponseWriter
185164 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
created xslt: solr.XSLTResponseWriter
185164 [http-bio-8080-exec-13] INFO
org.apache.solr.response.XSLTResponseWriter – xsltCacheLifetimeSeconds=5
185170 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – no
updateRequestProcessorChain defined as default, creating implicit default
185171 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /select: solr.SearchHandler
185173 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /query: solr.SearchHandler
185174 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /get: solr.RealTimeGetHandler
185175 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /browse: solr.SearchHandler
185177 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /update: solr.UpdateRequestHandler
185178 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /update/json: solr.UpdateRequestHandler
185179 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /update/csv: solr.UpdateRequestHandler
185179 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.extraction.ExtractingRequestHandler
185180 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /update/extract: solr.extraction.ExtractingRequestHandler
185180 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.FieldAnalysisRequestHandler
185181 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /analysis/field: solr.FieldAnalysisRequestHandler
185181 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
185182 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /analysis/document: solr.DocumentAnalysisRequestHandler
185183 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /admin/: solr.admin.AdminHandlers
185184 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /admin/ping: solr.PingRequestHandler
185185 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /debug/dump: solr.DumpRequestHandler
185186 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /replication: solr.ReplicationHandler
185187 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.SearchHandler
185187 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /spell: solr.SearchHandler
185188 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.SearchHandler
185188 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /suggest: solr.SearchHandler
185189 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.SearchHandler
185213 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /tvrh: solr.SearchHandler
185214 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.SearchHandler
185214 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /terms: solr.SearchHandler
185215 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– adding lazy requestHandler: solr.SearchHandler
185215 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
– created /elevate: solr.SearchHandler
185217 [http-bio-8080-exec-13] INFO
org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60
185218 [http-bio-8080-exec-13] INFO
org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60
185218 [http-bio-8080-exec-13] INFO
org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60
185223 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – Hard
AutoCommit: if uncommited for 15000ms;
185224 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – Soft
AutoCommit: disabled
185225 [http-bio-8080-exec-13] INFO
org.apache.solr.core.CachingDirectoryFactory – return new directory for
/ebs-data/solr/data/index
186227 [http-bio-8080-exec-13] INFO
org.apache.solr.core.CachingDirectoryFactory – looking to close
/ebs-data/solr/data/index
[CachedDir<<refCount=0;path=/ebs-data/solr/data/index;done=true>>]
186228 [http-bio-8080-exec-13] INFO
org.apache.solr.core.CachingDirectoryFactory – Closing directory:
/ebs-data/solr/data/index
186236 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
[realtimecollection_shard1_replica1] CLOSING SolrCore
org.apache.solr.core.SolrCore@810a57c
186243 [http-bio-8080-exec-13] INFO org.apache.solr.update.UpdateHandler
– closing DirectUpdateHandler2{commits=0,autocommit
maxTime=15000ms,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
186243 [http-bio-8080-exec-13] INFO org.apache.solr.update.SolrCoreState
– Closing SolrCoreState
186244 [http-bio-8080-exec-13] INFO
org.apache.solr.update.DefaultSolrCoreState – SolrCoreState ref count has
reached 0 - closing IndexWriter
186244 [http-bio-8080-exec-13] INFO
org.apache.solr.update.DefaultSolrCoreState – closing IndexWriter with
IndexWriterCloser
186245 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
[realtimecollection_shard1_replica1] Closing main searcher on request.
186246 [http-bio-8080-exec-13] INFO
org.apache.solr.core.CachingDirectoryFactory – Closing
NRTCachingDirectoryFactory - 1 directories currently being tracked
186247 [http-bio-8080-exec-13] INFO
org.apache.solr.core.CachingDirectoryFactory – looking to close
/ebs-data/solr/data
[CachedDir<<refCount=0;path=/ebs-data/solr/data;done=false>>]
186248 [http-bio-8080-exec-13] INFO
org.apache.solr.core.CachingDirectoryFactory – Closing directory:
/ebs-data/solr/data
186252 [http-bio-8080-exec-13] ERROR org.apache.solr.core.CoreContainer –
Unable to create core: realtimecollection_shard1_replica1
org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:571)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:200)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838)
... 24 more
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain
timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock:
java.nio.channels.OverlappingFileLockException
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1513)
... 26 more
Caused by: java.nio.channels.OverlappingFileLockException
at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1017)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:1154)
at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148)
at org.apache.lucene.store.Lock.obtain(Lock.java:100)
... 32 more
186334 [localhost-startStop-1-EventThread] INFO
org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
/overseer/queue state: SyncConnected type NodeChildrenChanged
186336 [http-bio-8080-exec-13] ERROR org.apache.solr.core.SolrCore –
org.apache.solr.common.SolrException: Error CREATEing SolrCore
'realtimecollection_shard1_replica1': Unable to create core:
realtimecollection_shard1_replica1 Caused by: null
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:608)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:200)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.solr.common.SolrException: Unable to create core:
realtimecollection_shard1_replica1
at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:571)
... 21 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
... 22 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838)
... 24 more
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain
timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock:
java.nio.channels.OverlappingFileLockException
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1513)
... 26 more
Caused by: java.nio.channels.OverlappingFileLockException
at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1017)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:1154)
at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148)
at org.apache.lucene.store.Lock.obtain(Lock.java:100)
... 32 more
186337 [http-bio-8080-exec-13] INFO
org.apache.solr.servlet.SolrDispatchFilter – [admin] webapp=null
path=/admin/cores
params={numShards=1&shard=shard1&collection.configName=collection1&name=realtimecollection_shard1_replica1&action=CREATE&collection=realtimecollection&wt=javabin&qt=/admin/cores&version=2}
status=400 QTime=7895
186487 [Thread-2] INFO org.apache.solr.common.cloud.ZkStateReader –
Updating cloud state from ZooKeeper...
186554 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
ERROR org.apache.solr.cloud.OverseerCollectionProcessor – Error from
shard: http://SERVERADDRESS:8080/solr
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error
CREATEing SolrCore 'realtimecollection_shard1_replica1': Unable to create
core: realtimecollection_shard1_replica1 Caused by: null
at
org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:157)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
186555 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
INFO org.apache.solr.cloud.OverseerCollectionProcessor – Finished create
command on all shards for collection: realtimecollection
186698 [localhost-startStop-1-EventThread] INFO
org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
/overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type
NodeDataChanged
186787 [localhost-startStop-1-EventThread] INFO
org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
/overseer/collection-queue-work state: SyncConnected type
NodeChildrenChanged
186792 [http-bio-8080-exec-9] INFO
org.apache.solr.servlet.SolrDispatchFilter – [admin] webapp=null
path=/admin/collections
params={numShards=1&collection.configName=collection1&createNodeSet=SERVERADDRESS:8080_solr&name=realtimecollection&action=CREATE}
status=0 QTime=11206
186795 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
INFO org.apache.solr.cloud.OverseerCollectionProcessor – Overseer
Collection Processor: Message
id:/overseer/collection-queue-work/qn-0000000000 complete,
response:{failure={null=org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
CREATEing SolrCore 'realtimecollection_shard1_replica1': Unable to create
core: realtimecollection_shard1_replica1 Caused by: null}}
186842 [localhost-startStop-1-EventThread] INFO
org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
/overseer/queue state: SyncConnected type NodeChildrenChanged
186908 [localhost-startStop-1-EventThread] INFO
org.apache.solr.common.cloud.ZkStateReader – A cluster state change:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
Re: SOLR 4.8 Collections API Create Collection Fails on Tomcat
Posted by cpalm <cp...@gmail.com>.
Shawn,
I agree the data dir concept doesn't make much sense in the context of
collections.
With the config I had setup the solr home was in a different file mount, so
I had to change my solr home
filemount location to get the data dir to locate where I wanted it once the
data dir option was removed.
>From the post above it sounds like it is supported with some lesser known
options, I will look at those to see if it gives any more flexibility in
this scenario.
The solr.xml I mentioned above was not the one that controls the cores.
In my solr.solr.home=/ebs-data/solr/conf folder there is still a solr.xml
that reflects the original core and the newly added core from the
collections api.
Thanks,
Chris
--
View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-8-Collections-API-Create-Collection-Fails-on-Tomcat-tp4137456p4137756.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR 4.8 Collections API Create Collection Fails on Tomcat
Posted by Shawn Heisey <so...@elyograg.org>.
On 5/21/2014 3:15 PM, cpalm wrote:
> Yes we use that to force the data dir to be someplace other than solr.home.
> Removing the data dir field just puts the data in solr home, which isn't
> desirable.
> solr home is specified in conf/Catalina/localhost/solr.xml
> <Context docBase="/etc/solr/solr.war" debug="0" crossContext="true">
> <Environment name="solr/home" type="java.lang.String" value="/etc/solr"
> override="true"/>
> </Context>
>
> Is there any way to use the collections api with a data.dir set?
The default location for the dataDir is ./data, relative to the core's
instanceDir, which itself is normally relative to solr home. It doesn't
make any sense to set the dataDir with the collections API, because the
directory would have to be different for every core the API call
creates. You could possibly argue that you want to do a relative
directory here, but I don't really see the point of that either.
With SolrCloud, the only things that will be in your instanceDir will be
the data directory and one or more .properties files, so there is not a
compelling argument for separating instanceDir from dataDir. With a
non-cloud install, you would have a conf directory as well, and it can
make sense to keep that separate ... but with SolrCloud, the contents
that would normally be in the conf directory are in Zookeeper instead.
Here's a directory listing of an instanceDir on a 4.2.1 SolrCloud
install. The solr home here is /index/mbsolr4, the core is named
uniphoto_shard1_replica1, because the collection is named uniphoto:
[root@mbsolr1a uniphoto_shard1_replica1]# pwd
/index/mbsolr4/uniphoto_shard1_replica1
[root@mbsolr1a uniphoto_shard1_replica1]# ls -al
total 12
drwxr-xr-x 3 solr solr 4096 Apr 10 2013 .
drwxr-xr-x 11 solr solr 4096 Feb 25 14:42 ..
drwxr-xr-x 4 solr solr 4096 Nov 30 10:26 data
If you don't have a solr.xml file (as you mentioned in a later email)
then Solr will not be running in multicore mode, and I have to wonder
whether SolrCloud will function correctly in that mode. I would not
expect it to work right.
Thanks,
Shawn
Re: SOLR 4.8 Collections API Create Collection Fails on Tomcat
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
You can actually specify a dataDir in the Collection CREATE API by passing
the property.dataDir=/your/data/dir parameter.
It is not very well documented but it is possible.
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api1
On Thu, May 22, 2014 at 4:28 AM, cpalm <cp...@gmail.com> wrote:
> What I did to finally get this working was remove the
> Catalina/localhost/solr.xml which was allowing the binary to be run from a
> different location,
> I made sure the solr.solr.home=/ebs-data/solr/conf was set to the file
> mount
> I wanted and I then had
> to remove the Dsolr.data.dir=/ebs-data/solr/data " argument.
> Once I did that, I can get collections to create successfully from the
> collections api which now show up under /ebs-data/solr/conf.
>
> In short it seems that the solr.data.dir argument can't be used with the
> collections API.
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SOLR-4-8-Collections-API-Create-Collection-Fails-on-Tomcat-tp4137456p4137506.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Regards,
Shalin Shekhar Mangar.
Re: SOLR 4.8 Collections API Create Collection Fails on Tomcat
Posted by cpalm <cp...@gmail.com>.
What I did to finally get this working was remove the
Catalina/localhost/solr.xml which was allowing the binary to be run from a
different location,
I made sure the solr.solr.home=/ebs-data/solr/conf was set to the file mount
I wanted and I then had
to remove the Dsolr.data.dir=/ebs-data/solr/data " argument.
Once I did that, I can get collections to create successfully from the
collections api which now show up under /ebs-data/solr/conf.
In short it seems that the solr.data.dir argument can't be used with the
collections API.
--
View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-8-Collections-API-Create-Collection-Fails-on-Tomcat-tp4137456p4137506.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR 4.8 Collections API Create Collection Fails on Tomcat
Posted by cpalm <cp...@gmail.com>.
Joel,
Yes we use that to force the data dir to be someplace other than solr.home.
Removing the data dir field just puts the data in solr home, which isn't
desirable.
solr home is specified in conf/Catalina/localhost/solr.xml
<Context docBase="/etc/solr/solr.war" debug="0" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="/etc/solr"
override="true"/>
</Context>
Is there any way to use the collections api with a data.dir set?
Thanks,
Chris
--
View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-8-Collections-API-Create-Collection-Fails-on-Tomcat-tp4137456p4137484.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR 4.8 Collections API Create Collection Fails on Tomcat
Posted by Joel Bernstein <jo...@gmail.com>.
Chris,
I wasn't able to reproduce the error with a stock install of the
HelioSearch Distribution for Solr (HDS). HDS runs under Tomcat.
Looking at the stack trace this looks like the problem:
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain
timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock:
java.nio.channels.OverlappingFileLockException
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
It looks like the new collection is using the same data dir as an existing
collection. This startup param might be the issue:
export JAVA_OPTS="$JAVA_OPTS -Dsolr.data.dir=/ebs-data/solr/data
at
Joel Bernstein
Search Engineer at Heliosearch
On Wed, May 21, 2014 at 2:08 PM, christopher palm <cp...@gmail.com> wrote:
> Hello,
> I am trying to use the collections API to add in a secondary collection
> similar to what is described in this article
> http://heliosearch.org/solrcloud-assigning-nodes-machines/
> It works fine on a jetty setup where bootstrap config dir is specified.
> When I try to run this on a tomcat solr that has the example configs
> already in zookeeper from an upconfig it fails
>
>
> http://SERVERADDRESS:8080/solr/admin/collections?action=CREATE&name=realtimecollection&collection.configName=collection1&numShards=1&createNodeSet=SERVERADDRESS:8080_solr
>
> ": Unable to create core: realtimecollection_shard1_replica1 Caused by:
> null"
>
> Any ideas why this fails?
>
>
> From the logs it seems to be trying to create this in the same folder as
> the solr 4.8 war.
>
> Below are the logs I see and the startup config arguments I am passing into
> tomcat/SOLR.
>
> Thanks,
> Chris
>
> export SOLR_HOME=/ebs-data/bin/solr
> export JAVA_OPTS="$JAVA_OPTS -DzkHost=ZOOKEEPERCLUSTERADDRESS/app/test"
> export JAVA_OPTS="$JAVA_OPTS -DnumShards=2"
> export JAVA_OPTS="$JAVA_OPTS -Dcollection.configName=collection1 "
> export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/ebs-data/solr/conf "
> export JAVA_OPTS="$JAVA_OPTS -Dsolr.data.dir=/ebs-data/solr/data "
>
>
> – A cluster state change: WatchedEvent state:SyncConnected
> type:NodeDataChanged path:/clusterstate.json, has occurred - updating...
> (live nodes size: 1)
> 176300 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
> INFO org.apache.solr.cloud.OverseerCollectionProcessor – Creating
> SolrCores for new collection, shardNames [shard1] , replicationFactor : 1
> 176300 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
> INFO org.apache.solr.cloud.OverseerCollectionProcessor – Creating shard
> realtimecollection_shard1_replica1 as part of slice shard1 of collection
> realtimecollection on SERVERADDRESS:8080_solr
> 178443 [http-bio-8080-exec-13] INFO
> org.apache.solr.handler.admin.CoreAdminHandler – core create command
>
> numShards=1&shard=shard1&collection.configName=collection1&name=realtimecollection_shard1_replica1&action=CREATE&collection=realtimecollection&wt=javabin&qt=/admin/cores&version=2
> 178540 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
> publishing core=realtimecollection_shard1_replica1 state=down
> collection=realtimecollection
> 178568 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
> /overseer/queue state: SyncConnected type NodeChildrenChanged
> 178569 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
> waiting to find shard id in clusterstate for
> realtimecollection_shard1_replica1
> 178641 [Thread-2] INFO org.apache.solr.common.cloud.ZkStateReader –
> Updating cloud state from ZooKeeper...
> 178667 [Thread-2] INFO org.apache.solr.cloud.Overseer – Update state
> numShards=1 message={
> "operation":"state",
> "state":"down",
> "base_url":"http://SERVERADDRESS:8080/solr",
> "core":"realtimecollection_shard1_replica1",
> "roles":null,
> "node_name":"SERVERADDRESS:8080_solr",
> "shard":"shard1",
> "collection":"realtimecollection",
> "numShards":"1",
>
>
> "core_node_name":"SERVERADDRESS:8080_solr_realtimecollection_shard1_replica1"}
> 178771 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
> /overseer/queue state: SyncConnected type NodeChildrenChanged
> 178798 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.common.cloud.ZkStateReader – A cluster state change:
> WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> 179572 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.SolrXMLCoresLocator – Persisted core descriptions to
> /etc/solr/solr.xml
> 179573 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
> Check for collection zkNode:realtimecollection
> 179600 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController –
> Collection zkNode exists
> 179600 [http-bio-8080-exec-13] INFO
> org.apache.solr.common.cloud.ZkStateReader – Load collection config
> from:/collections/realtimecollection
> 179654 [http-bio-8080-exec-13] INFO
> org.apache.solr.common.cloud.ZkStateReader –
> path=/collections/realtimecollection configName=collection1 specified
> config exists in ZooKeeper
> 179655 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.SolrResourceLoader – new SolrResourceLoader for
> directory: '/etc/solr/realtimecollection_shard1_replica1/'
> 179900 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrConfig –
> Adding specified lib dirs to ClassLoader
> 179901 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../contrib/extraction/lib (resolved as:
>
> /etc/solr/realtimecollection_shard1_replica1/../../../contrib/extraction/lib).
> 179902 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../dist/ (resolved as:
> /etc/solr/realtimecollection_shard1_replica1/../../../dist).
> 179902 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../contrib/clustering/lib/ (resolved as:
>
> /etc/solr/realtimecollection_shard1_replica1/../../../contrib/clustering/lib).
> 179903 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../dist/ (resolved as:
> /etc/solr/realtimecollection_shard1_replica1/../../../dist).
> 179903 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../contrib/langid/lib/ (resolved as:
> /etc/solr/realtimecollection_shard1_replica1/../../../contrib/langid/lib).
> 179904 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../dist/ (resolved as:
> /etc/solr/realtimecollection_shard1_replica1/../../../dist).
> 179904 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../contrib/velocity/lib (resolved as:
>
> /etc/solr/realtimecollection_shard1_replica1/../../../contrib/velocity/lib).
> 179904 [http-bio-8080-exec-13] WARN
> org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory
> to add to classloader: ../../../dist/ (resolved as:
> /etc/solr/realtimecollection_shard1_replica1/../../../dist).
> 180011 [http-bio-8080-exec-13] INFO org.apache.solr.update.SolrIndexConfig
> – IndexWriter infoStream solr logging is enabled
> 180038 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrConfig –
> Using Lucene MatchVersion: LUCENE_48
> 180185 [http-bio-8080-exec-13] INFO org.apache.solr.core.Config – Loaded
> SolrConfig: solrconfig.xml
> 180285 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema –
> Reading Solr Schema from schema.xml
> 180524 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema –
> [realtimecollection_shard1_replica1] Schema name=example
> 182144 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema –
> unique key field: id
> 182147 [http-bio-8080-exec-13] INFO org.apache.solr.rest.RestManager –
> Registered ManagedResource impl
> org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path
> /schema/analysis/stopwords/english
> 182148 [http-bio-8080-exec-13] INFO org.apache.solr.rest.RestManager –
> Registered ManagedResource impl
>
> org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager
> for path /schema/analysis/synonyms/english
> 183114 [http-bio-8080-exec-13] INFO
> org.apache.solr.schema.FileExchangeRateProvider – Reloading exchange
> rates from file currency.xml
> 183175 [http-bio-8080-exec-13] INFO
> org.apache.solr.schema.FileExchangeRateProvider – Reloading exchange
> rates from file currency.xml
> 185157 [http-bio-8080-exec-13] INFO org.apache.solr.core.CoreContainer –
> Creating SolrCore 'realtimecollection_shard1_replica1' using configuration
> from collection realtimecollection
> 185158 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> solr.NRTCachingDirectoryFactory
> 185158 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> [realtimecollection_shard1_replica1] Opening new SolrCore at
> /etc/solr/realtimecollection_shard1_replica1/, dataDir=/ebs-data/solr/data/
> 185159 [http-bio-8080-exec-13] INFO org.apache.solr.core.JmxMonitoredMap
> – JMX monitoring is enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@1de10458
> 185160 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> [realtimecollection_shard1_replica1] Added SolrEventListener for
> newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
> 185160 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> [realtimecollection_shard1_replica1] Added SolrEventListener for
> firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static
> firstSearcher warming in solrconfig.xml}]}
> 185161 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.CachingDirectoryFactory – return new directory for
> /ebs-data/solr/data
> 185162 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – New
> index directory detected: old=null new=/ebs-data/solr/data/index/
> 185163 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> created json: solr.JSONResponseWriter
> 185163 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> adding lazy queryResponseWriter: solr.VelocityResponseWriter
> 185164 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> created velocity: solr.VelocityResponseWriter
> 185164 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> created xslt: solr.XSLTResponseWriter
> 185164 [http-bio-8080-exec-13] INFO
> org.apache.solr.response.XSLTResponseWriter – xsltCacheLifetimeSeconds=5
> 185170 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – no
> updateRequestProcessorChain defined as default, creating implicit default
> 185171 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /select: solr.SearchHandler
> 185173 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /query: solr.SearchHandler
> 185174 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /get: solr.RealTimeGetHandler
> 185175 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /browse: solr.SearchHandler
> 185177 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /update: solr.UpdateRequestHandler
> 185178 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /update/json: solr.UpdateRequestHandler
> 185179 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /update/csv: solr.UpdateRequestHandler
> 185179 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.extraction.ExtractingRequestHandler
> 185180 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /update/extract: solr.extraction.ExtractingRequestHandler
> 185180 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.FieldAnalysisRequestHandler
> 185181 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /analysis/field: solr.FieldAnalysisRequestHandler
> 185181 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
> 185182 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /analysis/document: solr.DocumentAnalysisRequestHandler
> 185183 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /admin/: solr.admin.AdminHandlers
> 185184 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /admin/ping: solr.PingRequestHandler
> 185185 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /debug/dump: solr.DumpRequestHandler
> 185186 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /replication: solr.ReplicationHandler
> 185187 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.SearchHandler
> 185187 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /spell: solr.SearchHandler
> 185188 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.SearchHandler
> 185188 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /suggest: solr.SearchHandler
> 185189 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.SearchHandler
> 185213 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /tvrh: solr.SearchHandler
> 185214 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.SearchHandler
> 185214 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /terms: solr.SearchHandler
> 185215 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – adding lazy requestHandler: solr.SearchHandler
> 185215 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers
> – created /elevate: solr.SearchHandler
> 185217 [http-bio-8080-exec-13] INFO
> org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60
> 185218 [http-bio-8080-exec-13] INFO
> org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60
> 185218 [http-bio-8080-exec-13] INFO
> org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60
> 185223 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – Hard
> AutoCommit: if uncommited for 15000ms;
> 185224 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – Soft
> AutoCommit: disabled
> 185225 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.CachingDirectoryFactory – return new directory for
> /ebs-data/solr/data/index
> 186227 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.CachingDirectoryFactory – looking to close
> /ebs-data/solr/data/index
> [CachedDir<<refCount=0;path=/ebs-data/solr/data/index;done=true>>]
> 186228 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.CachingDirectoryFactory – Closing directory:
> /ebs-data/solr/data/index
> 186236 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> [realtimecollection_shard1_replica1] CLOSING SolrCore
> org.apache.solr.core.SolrCore@810a57c
> 186243 [http-bio-8080-exec-13] INFO org.apache.solr.update.UpdateHandler
> – closing DirectUpdateHandler2{commits=0,autocommit
> maxTime=15000ms,autocommits=0,soft
>
> autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
> 186243 [http-bio-8080-exec-13] INFO org.apache.solr.update.SolrCoreState
> – Closing SolrCoreState
> 186244 [http-bio-8080-exec-13] INFO
> org.apache.solr.update.DefaultSolrCoreState – SolrCoreState ref count has
> reached 0 - closing IndexWriter
> 186244 [http-bio-8080-exec-13] INFO
> org.apache.solr.update.DefaultSolrCoreState – closing IndexWriter with
> IndexWriterCloser
> 186245 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore –
> [realtimecollection_shard1_replica1] Closing main searcher on request.
> 186246 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.CachingDirectoryFactory – Closing
> NRTCachingDirectoryFactory - 1 directories currently being tracked
> 186247 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.CachingDirectoryFactory – looking to close
> /ebs-data/solr/data
> [CachedDir<<refCount=0;path=/ebs-data/solr/data;done=false>>]
> 186248 [http-bio-8080-exec-13] INFO
> org.apache.solr.core.CachingDirectoryFactory – Closing directory:
> /ebs-data/solr/data
> 186252 [http-bio-8080-exec-13] ERROR org.apache.solr.core.CoreContainer –
> Unable to create core: realtimecollection_shard1_replica1
> org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
> at
>
> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:571)
> at
>
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:200)
> at
>
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186)
> at
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
>
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> at
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
>
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838)
> ... 24 more
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain
> timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock:
> java.nio.channels.OverlappingFileLockException
> at org.apache.lucene.store.Lock.obtain(Lock.java:89)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
> at
>
> org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267)
> at
>
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1513)
> ... 26 more
> Caused by: java.nio.channels.OverlappingFileLockException
> at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
> at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
> at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1017)
> at java.nio.channels.FileChannel.tryLock(FileChannel.java:1154)
> at
> org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148)
> at org.apache.lucene.store.Lock.obtain(Lock.java:100)
> ... 32 more
> 186334 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
> /overseer/queue state: SyncConnected type NodeChildrenChanged
> 186336 [http-bio-8080-exec-13] ERROR org.apache.solr.core.SolrCore –
> org.apache.solr.common.SolrException: Error CREATEing SolrCore
> 'realtimecollection_shard1_replica1': Unable to create core:
> realtimecollection_shard1_replica1 Caused by: null
> at
>
> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:608)
> at
>
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:200)
> at
>
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186)
> at
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
>
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> at
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
>
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: org.apache.solr.common.SolrException: Unable to create core:
> realtimecollection_shard1_replica1
> at
> org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568)
> at
>
> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:571)
> ... 21 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
> ... 22 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838)
> ... 24 more
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain
> timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock:
> java.nio.channels.OverlappingFileLockException
> at org.apache.lucene.store.Lock.obtain(Lock.java:89)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
> at
>
> org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267)
> at
>
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1513)
> ... 26 more
> Caused by: java.nio.channels.OverlappingFileLockException
> at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
> at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
> at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1017)
> at java.nio.channels.FileChannel.tryLock(FileChannel.java:1154)
> at
> org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148)
> at org.apache.lucene.store.Lock.obtain(Lock.java:100)
> ... 32 more
>
> 186337 [http-bio-8080-exec-13] INFO
> org.apache.solr.servlet.SolrDispatchFilter – [admin] webapp=null
> path=/admin/cores
>
> params={numShards=1&shard=shard1&collection.configName=collection1&name=realtimecollection_shard1_replica1&action=CREATE&collection=realtimecollection&wt=javabin&qt=/admin/cores&version=2}
> status=400 QTime=7895
> 186487 [Thread-2] INFO org.apache.solr.common.cloud.ZkStateReader –
> Updating cloud state from ZooKeeper...
> 186554 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
> ERROR org.apache.solr.cloud.OverseerCollectionProcessor – Error from
> shard: http://SERVERADDRESS:8080/solr
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error
> CREATEing SolrCore 'realtimecollection_shard1_replica1': Unable to create
> core: realtimecollection_shard1_replica1 Caused by: null
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
> at
>
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:157)
> at
>
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> 186555 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
> INFO org.apache.solr.cloud.OverseerCollectionProcessor – Finished create
> command on all shards for collection: realtimecollection
> 186698 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
> /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type
> NodeDataChanged
> 186787 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
> /overseer/collection-queue-work state: SyncConnected type
> NodeChildrenChanged
> 186792 [http-bio-8080-exec-9] INFO
> org.apache.solr.servlet.SolrDispatchFilter – [admin] webapp=null
> path=/admin/collections
>
> params={numShards=1&collection.configName=collection1&createNodeSet=SERVERADDRESS:8080_solr&name=realtimecollection&action=CREATE}
> status=0 QTime=11206
> 186795 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000]
> INFO org.apache.solr.cloud.OverseerCollectionProcessor – Overseer
> Collection Processor: Message
> id:/overseer/collection-queue-work/qn-0000000000 complete,
>
> response:{failure={null=org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'realtimecollection_shard1_replica1': Unable to create
> core: realtimecollection_shard1_replica1 Caused by: null}}
> 186842 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path:
> /overseer/queue state: SyncConnected type NodeChildrenChanged
> 186908 [localhost-startStop-1-EventThread] INFO
> org.apache.solr.common.cloud.ZkStateReader – A cluster state change:
> WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>