You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Guylaine BASSETTE <gu...@francelabs.com> on 2023/06/01 08:45:26 UTC

Reading a document in Transfo Connector: Utility Classes

Hello,

I would like to contribute with some Utility Classes whose purpose is 
described below.

When you need to browse a document in a Transformation Connector, you 
have to store its stream, because after reading, it can't be read again 
at the Output Connector for Solr indexing.

I have created utility classes to store content of a document for 
browsing in a Transformation Connector, because each connector has 
currently its own way of doing it.
A build method automatically chooses the most suitable way to store data 
read thanks to the document size passed to the method: memory storage or 
temporary file storage. The max size for memory storage is a constant 
fixed to 65536 Bytes.

Here's an example:

DestinationStorage will be in memory or a temporary file ( 
File./createTempFile/(prefix, "tmp") ).

It is useful for many Transformation Connectors and is already in use in 
ours, where it's doing well.


-- 

Best Regards,
Guylaine

France Labs – Your knowledge, now
Datafari Enterprise Search – Découvrez la version 5 / Discover our version 5
www.datafari.com <http://www.datafari.com>