You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "VAIBHAV SHUKLA shuklavaibhavkj@yahoo.in" <va...@gmail.com> on 2019/03/14 07:13:25 UTC
Solr collection indexed to pdf in hdfs throws error during solr
restart
When I restart Solr it throws the following error. Solr collection indexed to pdf in hdfs throws error during solr restart.
Error
java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: Unable to create core [PDFIndex]
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.core.CoreContainer.lambda$load$6(CoreContainer.java:594)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Unable to create core [PDFIndex]
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:966)
at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
... 5 more
Caused by: org.apache.solr.common.SolrException: Index dir 'hdfs://192.168.1.16:8020/PDFIndex/data/index/' of core 'PDFIndex' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: hdfs
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950)
... 7 more
Caused by: org.apache.lucene.store.LockObtainFailedException: Index dir 'hdfs://192.168.1.16:8020/PDFIndex/data/index/' of core 'PDFIndex' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: hdfs
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:712)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:923)
... 9 more
2018-12-22 07:55:13.431 ERROR (OldIndexDirectoryCleanupThreadForCore-PDFIndex) [ x:PDFIndex] o.a.s.c.HdfsDirectoryFactory Error checking for old index directories to clean-up.
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808)
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2083)
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2069)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:791)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1517)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1557)
at org.apache.solr.core.HdfsDirectoryFactory.cleanupOldIndexDirectories(HdfsDirectoryFactory.java:546)
at org.apache.solr.core.SolrCore.lambda$cleanupOldIndexDirectories$19(SolrCore.java:3050)
at java.lang.Thread.run(Thread.java:748)
2018-12-22 07:55:13.433 ERROR (OldIndexDirectoryCleanupThreadForCore-PDFIndex) [ x:PDFIndex] o.a.s.c.SolrCore Failed to cleanup old index directories for core PDFIndex
java.lang.NullPointerException
at org.apache.solr.core.HdfsDirectoryFactory.cleanupOldIndexDirectories(HdfsDirectoryFactory.java:564)
at org.apache.solr.core.SolrCore.lambda$cleanupOldIndexDirectories$19(SolrCore.java:3050)
at java.lang.Thread.run(Thread.java:748)
I have created the collection in Solr which will index the pdf files and this collection is indexing all the pdf in HDFS.
Thanks & Regards
Vaibhav Shukla
Sent from Mail for Windows 10
Re: Solr collection indexed to pdf in hdfs throws error during solr
restart
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/14/2019 1:13 AM, VAIBHAV SHUKLA shuklavaibhavkj@yahoo.in wrote:
> When I restart Solr it throws the following error. Solr collection indexed to pdf in hdfs throws error during solr restart.
>
> Error
<snip>
> Caused by: org.apache.lucene.store.LockObtainFailedException: Index dir 'hdfs://192.168.1.16:8020/PDFIndex/data/index/' of core 'PDFIndex' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: hdfs
Solr has been shut down forcefully, so the lockfile is remaining in the
core's directory (which in your case is in HDFS). A graceful shutdown
would have deleted the lockfile.
What version of Solr, and what OS do you have it running on?
For a while now, on non-windows operating systems, the "stop" action in
the bin/solr script has waited up to 3 minutes for Solr to gracefully
shut down before forcefully killing it. This has eliminated most of
these problems when running on one of those operating systems.
On Windows, the bin\solr script is only waiting 5 seconds before
forcefully killing Solr, which used to happen on all operating systems.
This is extremely likely to cause problems like this. Fixing this on
Windows is on the radar, but in general we lack adept skill with
Windows, so it's not proceeding quickly.
I'm having trouble locating the issue for fixing the problem on Windows.
To fix it, find the "write.lock" file in your core's HDFS storage
location and delete it.
Thanks,
Shawn