You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Karl Wright (JIRA)" <ji...@apache.org> on 2014/07/01 10:30:25 UTC

[jira] [Created] (CONNECTORS-988) ManifoldCF pipeline should not dump small files/streams to disk

Karl Wright created CONNECTORS-988:
--------------------------------------

             Summary: ManifoldCF pipeline should not dump small files/streams to disk
                 Key: CONNECTORS-988
                 URL: https://issues.apache.org/jira/browse/CONNECTORS-988
             Project: ManifoldCF
          Issue Type: Improvement
          Components: Framework agents process
    Affects Versions: ManifoldCF 1.7
            Reporter: Karl Wright
            Assignee: Karl Wright
             Fix For: ManifoldCF 1.7


The ManifoldCF pipeline, when it splits, dumps data to disk.  Since most files are small, it should be more intelligent about this, and keep data in memory up to a certain size.

Turns out that TempFileInput and TempFileCharacterInput already support this functionality, so just changing over to support these core constructs would do the trick.



--
This message was sent by Atlassian JIRA
(v6.2#6252)