[jira] [Commented] (SOLR-8169) Need LockFactory impl that uses ZooKeeper as replacement for HdfsLockFactory

["Mark Miller (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-8169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991874#comment-14991874 ] 

Mark Miller commented on SOLR-8169:
-----------------------------------

I've got a simple impl. Sadly, ZK is a little problematic to store these locks I think though.

It would be nice to be able to use ephemeral nodes - so we prevent stale locks on a crash or something. That doesn't really tie well into the lifecycle of the IndexWriter though. It may help that we don't allow updates when not connected to ZK, but it's all a little tricky.

So I have a little test impl with no ephem nodes I can toss up when it's cleaned up a littlle. Which may not seem like much, but I don't think the HDFS filesystem actually works properly for a file shared lock the way it's being used in our HdfsLockFactory. This is something that at least works correctly.

> Need LockFactory impl that uses ZooKeeper as replacement for HdfsLockFactory
> ----------------------------------------------------------------------------
>
>                 Key: SOLR-8169
>                 URL: https://issues.apache.org/jira/browse/SOLR-8169
>             Project: Solr
>          Issue Type: New Feature
>          Components: Hadoop Integration
>            Reporter: Timothy Potter
>
> It would be good to have an option to use a ZooKeeper backed LockFactory implementation as a replacement for the HdfsLockFactory. FWIW - I've seen instances in Solr on YARN environments where the lock file doesn't get cleaned up correctly, which prevents using the index w/o some manual intervention.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org