[jira] [Commented] (CONNECTORS-781) Fault-Tolerant Setup for ManifoldCF Agent.

["Graeme Seaton (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CONNECTORS-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822841#comment-13822841 ] 

Graeme Seaton commented on CONNECTORS-781:
------------------------------------------

Would it possible for the Zookeeper ensemble to be aware of the number of threads/connections working on a particular element and acting as the 'global' tally for all clients which they (the clients)would then use as their throttle (or is that what you were saying)?

> Fault-Tolerant Setup for ManifoldCF Agent.
> ------------------------------------------
>
>                 Key: CONNECTORS-781
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-781
>             Project: ManifoldCF
>          Issue Type: Improvement
>          Components: Framework agents process, Framework core, Framework crawler agent
>    Affects Versions: ManifoldCF 1.5
>            Reporter: Swami Rajamohan
>            Assignee: Karl Wright
>              Labels: agents, crawler, fault-tolerance
>             Fix For: ManifoldCF 1.5
>
>
> It should be possible to setup ManifoldCF as a Fault-Tolerant infrastructure.
> The Agent component of ManifoldCF should support multiple instances of an agent crawling against a single crawl store, to be able to both distribute (share) the crawl load as well as to be able to pick up a request that gets abruptly terminated due to either partitioning of the instance/failure of the instance itself.
> Since there is a proposal to move to a store like Voldemort, it would be nice to be able to have a fault tolerant infrastructure.



--
This message was sent by Atlassian JIRA
(v6.1#6144)