You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "yuanyun.cn (JIRA)" <ji...@apache.org> on 2014/11/21 22:40:33 UTC
[jira] [Created] (SOLR-6776) Data lost when use SoftCommit and TLog
yuanyun.cn created SOLR-6776:
--------------------------------
Summary: Data lost when use SoftCommit and TLog
Key: SOLR-6776
URL: https://issues.apache.org/jira/browse/SOLR-6776
Project: Solr
Issue Type: Bug
Affects Versions: 4.10
Reporter: yuanyun.cn
Fix For: 4.10.3
We enabled update log and change autoCommit to some bigger value 10 mins.
After restart, we push one doc with softCommit=true
http://localhost:8983/solr/update?stream.body=<add><doc><field name="id">id1</field></doc></add>&softCommit=true
Then we kill the java process after a min.
After restart, Tlog failed to replay with following exception, and there is no data in solr.
6245 [coreLoadExecutor-5-thread-1] ERROR org.apache.solr.update.UpdateLog รป Failure to open existing log file (non fatal) E:\jeffery\src\apache\solr\4.10.2\solr-4.10.2\example\solr\collection1\data\t
log\tlog.0000000000000000000:org.apache.solr.common.SolrException: java.io.EOFException
at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:181)
at org.apache.solr.update.UpdateLog.init(UpdateLog.java:261)
at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:134)
at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)
at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:100)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:550)
at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:620)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:835)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:646)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException
at org.apache.solr.common.util.FastInputStream.readUnsignedByte(FastInputStream.java:73)
at org.apache.solr.common.util.FastInputStream.readInt(FastInputStream.java:216)
at org.apache.solr.update.TransactionLog.readHeader(TransactionLog.java:268)
at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:159)
... 19 more
Check the code: seems this is related with: org.apache.solr.update.processor.RunUpdateProcessor, in processCommit, it sets changesSinceCommit=false(even we are using softCommit)
So in finish, updateLog.finish will not be called.
public void finish() throws IOException {
if (changesSinceCommit && updateHandler.getUpdateLog() != null) {
updateHandler.getUpdateLog().finish(null);
}
super.finish();
}
To fix this issue: I have to change RunUpdateProcessor.processCommit like below:
if (!cmd.softCommit) {
changesSinceCommit = false;
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org