You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Mohit Anchlia <mo...@gmail.com> on 2017/07/31 20:07:35 UTC
Using FileInputFormat - org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
In trying to use this code I get the following error. Is it asking me to
implement additional interface?
streamEnv.readFile(format, args[0], FileProcessingMode.
*PROCESS_CONTINUOUSLY*, 2000).print();
[main] INFO com.s.flink.example.PDFInputFormat - Start streaming
[main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
does not contain a setter for field modificationTime
[main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
is not a valid POJO type because not all fields are valid POJO fields.
Re: Using FileInputFormat - org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
Posted by Mohit Anchlia <mo...@gmail.com>.
This was user induced problem - me. I wasn't calling streamenv.execute() :(
On Tue, Aug 1, 2017 at 1:29 PM, Mohit Anchlia <mo...@gmail.com>
wrote:
> This doesn't work even with TextInputFormat. Not sure what's wrong.
>
> On Tue, Aug 1, 2017 at 9:53 AM, Mohit Anchlia <mo...@gmail.com>
> wrote:
>
>> I don't see the print output.
>>
>> On Tue, Aug 1, 2017 at 2:08 AM, Fabian Hueske <fh...@gmail.com> wrote:
>>
>>> Hi Mohit,
>>>
>>> these are just INFO log statements that do not necessarily indicate a
>>> problem.
>>> Is the program working otherwise or do you observe other problems?
>>>
>>> Best, Fabian
>>>
>>> 2017-08-01 0:32 GMT+02:00 Mohit Anchlia <mo...@gmail.com>:
>>>
>>>> I even tried existing format but still same error:
>>>>
>>>> FileInputFormat<String> fileInputFormat = *new* TextInputFormat(*new*
>>>> Path(args[0]));
>>>>
>>>> fileInputFormat.setNestedFileEnumeration(*true*);
>>>>
>>>> streamEnv.readFile(fileInputFormat, args[0],
>>>>
>>>> FileProcessingMode.*PROCESS_CONTINUOUSLY*, 10000L).print();
>>>>
>>>>
>>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>>> does not contain a setter for field modificationTime
>>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - c
>>>>
>>>> On Mon, Jul 31, 2017 at 1:07 PM, Mohit Anchlia <mo...@gmail.com>
>>>> wrote:
>>>>
>>>>> In trying to use this code I get the following error. Is it asking me
>>>>> to implement additional interface?
>>>>>
>>>>> streamEnv.readFile(format, args[0], FileProcessingMode.
>>>>> *PROCESS_CONTINUOUSLY*, 2000).print();
>>>>>
>>>>>
>>>>> [main] INFO com.s.flink.example.PDFInputFormat - Start streaming
>>>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>>>> does not contain a setter for field modificationTime
>>>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>>>> is not a valid POJO type because not all fields are valid POJO fields.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Using FileInputFormat - org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
Posted by Mohit Anchlia <mo...@gmail.com>.
This doesn't work even with TextInputFormat. Not sure what's wrong.
On Tue, Aug 1, 2017 at 9:53 AM, Mohit Anchlia <mo...@gmail.com>
wrote:
> I don't see the print output.
>
> On Tue, Aug 1, 2017 at 2:08 AM, Fabian Hueske <fh...@gmail.com> wrote:
>
>> Hi Mohit,
>>
>> these are just INFO log statements that do not necessarily indicate a
>> problem.
>> Is the program working otherwise or do you observe other problems?
>>
>> Best, Fabian
>>
>> 2017-08-01 0:32 GMT+02:00 Mohit Anchlia <mo...@gmail.com>:
>>
>>> I even tried existing format but still same error:
>>>
>>> FileInputFormat<String> fileInputFormat = *new* TextInputFormat(*new*
>>> Path(args[0]));
>>>
>>> fileInputFormat.setNestedFileEnumeration(*true*);
>>>
>>> streamEnv.readFile(fileInputFormat, args[0],
>>>
>>> FileProcessingMode.*PROCESS_CONTINUOUSLY*, 10000L).print();
>>>
>>>
>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>> does not contain a setter for field modificationTime
>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - c
>>>
>>> On Mon, Jul 31, 2017 at 1:07 PM, Mohit Anchlia <mo...@gmail.com>
>>> wrote:
>>>
>>>> In trying to use this code I get the following error. Is it asking me
>>>> to implement additional interface?
>>>>
>>>> streamEnv.readFile(format, args[0], FileProcessingMode.
>>>> *PROCESS_CONTINUOUSLY*, 2000).print();
>>>>
>>>>
>>>> [main] INFO com.s.flink.example.PDFInputFormat - Start streaming
>>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>>> does not contain a setter for field modificationTime
>>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>>> is not a valid POJO type because not all fields are valid POJO fields.
>>>>
>>>
>>>
>>
>
Re: Using FileInputFormat - org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
Posted by Mohit Anchlia <mo...@gmail.com>.
I don't see the print output.
On Tue, Aug 1, 2017 at 2:08 AM, Fabian Hueske <fh...@gmail.com> wrote:
> Hi Mohit,
>
> these are just INFO log statements that do not necessarily indicate a
> problem.
> Is the program working otherwise or do you observe other problems?
>
> Best, Fabian
>
> 2017-08-01 0:32 GMT+02:00 Mohit Anchlia <mo...@gmail.com>:
>
>> I even tried existing format but still same error:
>>
>> FileInputFormat<String> fileInputFormat = *new* TextInputFormat(*new*
>> Path(args[0]));
>>
>> fileInputFormat.setNestedFileEnumeration(*true*);
>>
>> streamEnv.readFile(fileInputFormat, args[0],
>>
>> FileProcessingMode.*PROCESS_CONTINUOUSLY*, 10000L).print();
>>
>>
>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>> does not contain a setter for field modificationTime
>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - c
>>
>> On Mon, Jul 31, 2017 at 1:07 PM, Mohit Anchlia <mo...@gmail.com>
>> wrote:
>>
>>> In trying to use this code I get the following error. Is it asking me
>>> to implement additional interface?
>>>
>>> streamEnv.readFile(format, args[0], FileProcessingMode.
>>> *PROCESS_CONTINUOUSLY*, 2000).print();
>>>
>>>
>>> [main] INFO com.s.flink.example.PDFInputFormat - Start streaming
>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>> does not contain a setter for field modificationTime
>>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>>> is not a valid POJO type because not all fields are valid POJO fields.
>>>
>>
>>
>
Re: Using FileInputFormat - org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
Posted by Fabian Hueske <fh...@gmail.com>.
Hi Mohit,
these are just INFO log statements that do not necessarily indicate a
problem.
Is the program working otherwise or do you observe other problems?
Best, Fabian
2017-08-01 0:32 GMT+02:00 Mohit Anchlia <mo...@gmail.com>:
> I even tried existing format but still same error:
>
> FileInputFormat<String> fileInputFormat = *new* TextInputFormat(*new*
> Path(args[0]));
>
> fileInputFormat.setNestedFileEnumeration(*true*);
>
> streamEnv.readFile(fileInputFormat, args[0],
>
> FileProcessingMode.*PROCESS_CONTINUOUSLY*, 10000L).print();
>
>
> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
> does not contain a setter for field modificationTime
> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - c
>
> On Mon, Jul 31, 2017 at 1:07 PM, Mohit Anchlia <mo...@gmail.com>
> wrote:
>
>> In trying to use this code I get the following error. Is it asking me to
>> implement additional interface?
>>
>> streamEnv.readFile(format, args[0], FileProcessingMode.
>> *PROCESS_CONTINUOUSLY*, 2000).print();
>>
>>
>> [main] INFO com.s.flink.example.PDFInputFormat - Start streaming
>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>> does not contain a setter for field modificationTime
>> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
>> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
>> is not a valid POJO type because not all fields are valid POJO fields.
>>
>
>
Re: Using FileInputFormat - org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
Posted by Mohit Anchlia <mo...@gmail.com>.
I even tried existing format but still same error:
FileInputFormat<String> fileInputFormat = *new* TextInputFormat(*new* Path(
args[0]));
fileInputFormat.setNestedFileEnumeration(*true*);
streamEnv.readFile(fileInputFormat, args[0],
FileProcessingMode.*PROCESS_CONTINUOUSLY*, 10000L).print();
[main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
does not contain a setter for field modificationTime
[main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - c
On Mon, Jul 31, 2017 at 1:07 PM, Mohit Anchlia <mo...@gmail.com>
wrote:
> In trying to use this code I get the following error. Is it asking me to
> implement additional interface?
>
> streamEnv.readFile(format, args[0], FileProcessingMode.
> *PROCESS_CONTINUOUSLY*, 2000).print();
>
>
> [main] INFO com.s.flink.example.PDFInputFormat - Start streaming
> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
> does not contain a setter for field modificationTime
> [main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
> org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit
> is not a valid POJO type because not all fields are valid POJO fields.
>