SimpleSingleInputOperator

[Jaikit Jilka <jj...@leadferret.com>]

Hello,

I created an operator which extends SimlpeSingleInputOperator. I read .json files using this operator. But this operator does not read files continously i.e. it won't read files that are being downloaded in a particular folder in real time. So how do I make the operator read files as and when they are downloaded in the folder.

Thank you,

Jaikit Jilka

Re: SimpleSingleInputOperator

[Sanjay Pujare <sa...@datatorrent.com>]

You might be aware of com.datatorrent.lib.io.fs.AbstractFileInputOperator<T>  whose description says

This is the base implementation of a directory input operator, which scans a directory for files.
 * Files are then read and split into tuples, which are emitted.


Does this work for you? You can then feed that into a JsonParser.


On 11/10/16, 1:09 PM, "Jaikit Jilka" <jj...@leadferret.com> wrote:

    Hello,
    
    I created an operator which extends SimlpeSingleInputOperator. I read .json files using this operator. But this operator does not read files continously i.e. it won't read files that are being downloaded in a particular folder in real time. So how do I make the operator read files as and when they are downloaded in the folder.
    
    Thank you,
    
    Jaikit Jilka