[jira] [Created] (CONNECTORS-663) ManifoldCF needs the ability to not always check for deletion on a crawl

["Karl Wright (JIRA)" <ji...@apache.org>]

Karl Wright created CONNECTORS-663:
--------------------------------------

             Summary: ManifoldCF needs the ability to not always check for deletion on a crawl
                 Key: CONNECTORS-663
                 URL: https://issues.apache.org/jira/browse/CONNECTORS-663
             Project: ManifoldCF
          Issue Type: New Feature
          Components: Framework agents process
    Affects Versions: ManifoldCF 1.1.1
            Reporter: Karl Wright
            Assignee: Karl Wright
             Fix For: ManifoldCF next


The ManifoldCF framework's crawling model always brings the index in synch with the repository by the end of the job.  Unfortunately, for many repositories, the incremental nature of ManifoldCF is lost in part because deletion tracking is not done by the repository.  ManifoldCF could therefore benefit by the ability to have two different job run cycles: (1) A full run, as is done now, and (2) a partial run, which does not necessarily attempt to clean up deletions.  This of course only makes sense if subsequent job runs have the ability to do the deletion cleanup.

In principle, I believe this can can work but has significant implications in the following areas:

- Job states - there needs to be a new set of job states corresponding to which type of job run is selected;
- UI - there needs to be a way of telling ManifoldCF what kind of job run is desired;
- API - same problem as UI;
- Job scheduling; we need the ability to determine what kind of job run is done when, which also has schema implications



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira