You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Kayak28 <ka...@gmail.com> on 2019/08/28 10:01:55 UTC
Problem of Shutdown Process for Windows Server
Hello, Community:
I use Solr with Windows servers, and cannot shutdown Solr successfully.
When I try to stop Solr using solr.cmd, which is kicked from Windows Task
Manager, it "looks" like Solr stops without any problem.
Here "looks" means that at least log file that Solr wrote does not seem to
have any error.
(I pasted a piece of the log where I believe "success" at the end of this
email )
However, next time I start up the Solr, I face the error message that says
"Address already in use."
This problem happens occasionally, happens a different server at irregular
time/date.
So, I could not simulate the situation yet.
I wonder why Solr could not shutdown successfully.
If anyone of you has faced a similar incident or knows a solution, then it
is very helpful to share your bits of advice.
Any clue will be very appreciated.
*Environment*
OS: Windows Server 2012 R2
Java: Oracle JDK 1.8.0
Solr Version: 5.2.1
Solr Structures:15 Solr server, enabled to distributed search with sharding
(Not using SolrCloud)
Memory(Solr / physical) : 20GB/32GB
Index Size: around 300GB
*Logs*
INFO - 2019-05-25 21:06:15.996; [ ]
org.apache.solr.core.CachingDirectoryFactory; looking to close
D:\Documents\solr-home\collection1\data
[CachedDir<<refCount=0;path=D:\Documents\solr-home\collection1\data;done=false>>]
INFO - 2019-05-25 21:06:15.996; [ ]
org.apache.solr.core.CachingDirectoryFactory; Closing directory:
D:\Documents\solr-home\collection1\data
INFO - 2019-05-25 21:06:15.996; [ ]
org.apache.solr.core.CachingDirectoryFactory; looking to close
D:\Documents\solr-home\collection1\data\index
[CachedDir<<refCount=0;path=D:\Documents\solr-home\collection1\data\index;done=false>>]
INFO - 2019-05-25 21:06:15.996; [ ]
org.apache.solr.core.CachingDirectoryFactory; Closing directory:
D:\Documents\solr-home\collection1\data\index
INFO - 2019-05-25 21:06:16.199; [ ]
org.eclipse.jetty.server.handler.ContextHandler; Stopped
o.e.j.w.WebAppContext@4b9e13df
{/solr,file:/D:/Documents/solr/server/solr-webapp/webapp/,UNAVAILABLE}{/solr.war}
* Note: solr-home directory is the directory where I store Solr cores.
Sincerely,
Kaya Ota
Re: Problem of Shutdown Process for Windows Server
Posted by Kayak28 <ka...@gmail.com>.
Hello, Shawn and Community:
Thank you for a quick response, and giving useful information.
As far as I understand, the main cause of this problem is something like
"time out."
This time out could perhaps happen due to insufficient memory for our index
size, so waiting time is not enough to gracefully stop Solr.
And in the case time out happens, a user should have the right permission
to kill Solr process.
If the user is not eligible to kill it, Solr process remains to use its
port number.
As a result, when I start Solr again, it shows me "Address already in use."
The possible actions that I can do to resolve this issue are to have bigger
memory and to check the user's permission.
Do I understand what you emailed me correctly?
I believe the following lines of code will shutdown Solr forcefully(
because comment out says so )
These codes come from line 718-735 of solr.cmd file.
Do you think it can be better if I edit "timeout /T 5" to a greater number
so that waiting time becomes longer than 5 seconds?
Longer waiting time will help the system to shutdown Solr gracefully?
IF "%%x"=="0.0.0.0" (
set found_it=1
@echo Stopping Solr process %%N running on port %SOLR_PORT%
set /A STOP_PORT=%SOLR_PORT% - 1000
"%JAVA%" %SOLR_SSL_OPTS% -Djetty.home="%SOLR_SERVER_DIR%" -jar
"%SOLR_SERVER_DIR%\start.jar" "%SOLR_JETTY_CONFIG%" STOP.PORT=!STOP_PORT!
STOP.KEY=%STOP_KEY% --stop
del "%SOLR_TIP%"\bin\solr-%SOLR_PORT%.port
*timeout /T 5*
REM Kill it if it is still running after the graceful
For /f "tokens=2,5" %%j in ('netstat -nao ^| find "TCP " ^|
find ":%SOLR_PORT% "') do (
IF "%%N"=="%%k" (
for /f "delims=: tokens=1,2" %%a IN ("%%j") do (
IF "%%a"=="0.0.0.0" (
@echo Forcefully killing process %%N
* taskkill /f /PID %%N*
)
)
)
)
)
I need to check my Java version later, but I wonder why Java version should
be at least 1.8.0_40?
Again, thank you for your response.
Sincerely,
Kaya Ota
2019年8月29日(木) 0:33 Shawn Heisey <ap...@elyograg.org>:
> On 8/28/2019 4:01 AM, Kayak28 wrote:
> > I use Solr with Windows servers, and cannot shutdown Solr successfully.
> > When I try to stop Solr using solr.cmd, which is kicked from Windows Task
> > Manager, it "looks" like Solr stops without any problem.
> > Here "looks" means that at least log file that Solr wrote does not seem
> to
> > have any error.
> > (I pasted a piece of the log where I believe "success" at the end of this
> > email )
>
> The solr.cmd script will try to stop Solr gracefully, wait five seconds,
> and then forcibly terminate it. We have modified this operation in
> recent versions for operating systems other than windows, so that the
> bash script will wait up to three minutes for Solr to terminate
> gracefully before it is forcibly terminated. But this has not been done
> on Windows.
>
> > *Environment*
> > OS: Windows Server 2012 R2
> > Java: Oracle JDK 1.8.0
> > Solr Version: 5.2.1
>
> Which of the many Java 8 versions are you running? 1.8.0 is not
> specific enough. You should be running at least build 40, and something
> numbered above 100 would be better. The latest 1.8.0 versions of Oracle
> Java have a different license than earlier versions, something you might
> need to be aware of. Oracle is now requiring a paid license for any
> production use of their Java. Only development can be done for free.
>
> > Solr Structures:15 Solr server, enabled to distributed search with
> sharding
> > (Not using SolrCloud)
> > Memory(Solr / physical) : 20GB/32GB
> > Index Size: around 300GB
>
> You should probably have at least 128GB of total system memory for 300GB
> of index, and 256GB would be better. Assuming that there is no software
> other than Solr on this machine, you only have about 12GB of memory left
> to cache that 300GB of index data. If there is other software on the
> system, there will probably be even less memory available. This could
> cause Solr to be very slow to shut down.
>
> Maybe the user that's running the stop command doesn't have permission
> to forcibly terminate the Solr process. In which case you would have to
> wait for the graceful shutdown, and as I just mentioned, that could be
> very slow on your setup.
>
> Thanks,
> Shawn
>
Re: Problem of Shutdown Process for Windows Server
Posted by Shawn Heisey <ap...@elyograg.org>.
On 8/28/2019 4:01 AM, Kayak28 wrote:
> I use Solr with Windows servers, and cannot shutdown Solr successfully.
> When I try to stop Solr using solr.cmd, which is kicked from Windows Task
> Manager, it "looks" like Solr stops without any problem.
> Here "looks" means that at least log file that Solr wrote does not seem to
> have any error.
> (I pasted a piece of the log where I believe "success" at the end of this
> email )
The solr.cmd script will try to stop Solr gracefully, wait five seconds,
and then forcibly terminate it. We have modified this operation in
recent versions for operating systems other than windows, so that the
bash script will wait up to three minutes for Solr to terminate
gracefully before it is forcibly terminated. But this has not been done
on Windows.
> *Environment*
> OS: Windows Server 2012 R2
> Java: Oracle JDK 1.8.0
> Solr Version: 5.2.1
Which of the many Java 8 versions are you running? 1.8.0 is not
specific enough. You should be running at least build 40, and something
numbered above 100 would be better. The latest 1.8.0 versions of Oracle
Java have a different license than earlier versions, something you might
need to be aware of. Oracle is now requiring a paid license for any
production use of their Java. Only development can be done for free.
> Solr Structures:15 Solr server, enabled to distributed search with sharding
> (Not using SolrCloud)
> Memory(Solr / physical) : 20GB/32GB
> Index Size: around 300GB
You should probably have at least 128GB of total system memory for 300GB
of index, and 256GB would be better. Assuming that there is no software
other than Solr on this machine, you only have about 12GB of memory left
to cache that 300GB of index data. If there is other software on the
system, there will probably be even less memory available. This could
cause Solr to be very slow to shut down.
Maybe the user that's running the stop command doesn't have permission
to forcibly terminate the Solr process. In which case you would have to
wait for the graceful shutdown, and as I just mentioned, that could be
very slow on your setup.
Thanks,
Shawn