You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Noble Paul (JIRA)" <ji...@apache.org> on 2009/01/23 13:04:59 UTC
[jira] Created: (SOLR-978) Old files are not removed from slaves
after replication
Old files are not removed from slaves after replication
-------------------------------------------------------
Key: SOLR-978
URL: https://issues.apache.org/jira/browse/SOLR-978
Project: Solr
Issue Type: Bug
Components: replication (java)
Reporter: Noble Paul
Fix For: 1.4
The files in the slave's index is not removed even after they are removed from master.
The temp directories named index.xxxxxxx are not removed after use
The problem is that when we do a commit on the slave after replication is done. The commit does not re-open the IndexWriter. Therefore, the deletion policy does not take affect and older files are left as is. This can keep on building up. The only solution is to re-open the index writer.
A user has reported the issue http://markmail.org/thread/yw5n4dk2t5zbt5z5
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-978) Old files are not removed from slaves
after replication
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-978:
---------------------------------------
Attachment: SOLR-978.patch
Changes
# Add a forceOpenWriter method to DirectUpdateHandler2 which is called by SnapPuller on the slave to make sure IndexWriter is open. This is to make sure IndexDeletionPolicy gets a chance to cleanup older commit points.
# SnapPuller calls forceOpenWriter if core.getUpdateHandler is an instance of DirectUpdateHandler2.
# SnapPuller calls delTree on temporary files and folders in a finally block
I know 2 is not a very clean way. I thought of adding that method in UpdateHandler as a no-op and overriding it in DUH2. However that would mean that custom UpdateHandler implementations would silently ignore this call and we will not have a chance to warn them (which we can do with this patch). Also, DUH2 already has a protected openWriter method. However calling that method needs access to a protected lock object.
> Old files are not removed from slaves after replication
> -------------------------------------------------------
>
> Key: SOLR-978
> URL: https://issues.apache.org/jira/browse/SOLR-978
> Project: Solr
> Issue Type: Bug
> Components: replication (java)
> Reporter: Noble Paul
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: SOLR-978.patch
>
>
> The files in the slave's index is not removed even after they are removed from master.
> The temp directories named index.xxxxxxx are not removed after use
> The problem is that when we do a commit on the slave after replication is done. The commit does not re-open the IndexWriter. Therefore, the deletion policy does not take affect and older files are left as is. This can keep on building up. The only solution is to re-open the index writer.
> A user has reported the issue http://markmail.org/thread/yw5n4dk2t5zbt5z5
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-978) Old files are not removed from slaves
after replication
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar reassigned SOLR-978:
------------------------------------------
Assignee: Shalin Shekhar Mangar
> Old files are not removed from slaves after replication
> -------------------------------------------------------
>
> Key: SOLR-978
> URL: https://issues.apache.org/jira/browse/SOLR-978
> Project: Solr
> Issue Type: Bug
> Components: replication (java)
> Reporter: Noble Paul
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
>
> The files in the slave's index is not removed even after they are removed from master.
> The temp directories named index.xxxxxxx are not removed after use
> The problem is that when we do a commit on the slave after replication is done. The commit does not re-open the IndexWriter. Therefore, the deletion policy does not take affect and older files are left as is. This can keep on building up. The only solution is to re-open the index writer.
> A user has reported the issue http://markmail.org/thread/yw5n4dk2t5zbt5z5
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-978) Old files are not removed from slaves
after replication
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar resolved SOLR-978.
----------------------------------------
Resolution: Fixed
Committed revision 737550.
Thanks Jaco and Noble!
> Old files are not removed from slaves after replication
> -------------------------------------------------------
>
> Key: SOLR-978
> URL: https://issues.apache.org/jira/browse/SOLR-978
> Project: Solr
> Issue Type: Bug
> Components: replication (java)
> Reporter: Noble Paul
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: SOLR-978.patch
>
>
> The files in the slave's index is not removed even after they are removed from master.
> The temp directories named index.xxxxxxx are not removed after use
> The problem is that when we do a commit on the slave after replication is done. The commit does not re-open the IndexWriter. Therefore, the deletion policy does not take affect and older files are left as is. This can keep on building up. The only solution is to re-open the index writer.
> A user has reported the issue http://markmail.org/thread/yw5n4dk2t5zbt5z5
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.