You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Rinat <r....@cleverdata.ru> on 2017/10/23 13:16:27 UTC
How to test new sink
Hi !!!
I’ve just implemented a new sink, that extends functionality of existing BucketingSink, currently I’m trying to test functionality, that is related with timing.
My sink implements ProcessingTimeCallback, similarly with the original BucketingSink. I’m trying to inject TestProcessingTimeService to test
I discovered, that original tests using some kind of test templates for testing functions OneInputStreamOperatorTestHarness, extending which test impl of ProcessingTimeService is injected,
but this templates are in test scope of flink-streaming-java module, so it could’t be accessed by the external projects.
May be someone could help me and explain the better approach for testing sink functionality ?
Thx !
Re: How to test new sink
Posted by Timo Walther <tw...@apache.org>.
Yes, if you think you need better public test utilities. Feel free to
open an issue for it.
Timo
Am 10/23/17 um 5:32 PM schrieb Rinat:
> Timo, thx for your reply.
> I’m using gradle instead of maven, but I’ll look through the existing
> similar plugins for it.
>
> I don’t think, that sharing of external tests between other projects
> is a good idea, but it’s out of scope of current discussion.
> The main purpose of my request is to understand the existing best
> practices for testing functions, that uses *TimeService *and after
> this, if it’ll be necessary, create an issue with proposals.
>
> Thx.
>
>> On 23 Oct 2017, at 17:51, Timo Walther <twalthr@apache.org
>> <ma...@apache.org>> wrote:
>>
>> Hi Rinat,
>>
>> using one of the Flink test utilities is a good approach to test your
>> custom operators. But of course these classes might change in the
>> future.
>>
>> First of all, Flink is a open source project so you can just copy the
>> required classes. However, it should be possible to use the Flink
>> test resources like explained here [1]. Flink adds the `test-jar` goal.
>>
>> @Gary: You are working on the BucketingSink right now, right? Do you
>> have a suggestion?
>>
>> Regards,
>> Timo
>>
>>
>> [1]
>> https://stackoverflow.com/questions/29653914/how-can-i-reference-unit-test-classes-of-a-maven-dependency-in-my-java-project
>>
>>
>> Am 10/23/17 um 3:16 PM schrieb Rinat:
>>> Hi !!!
>>>
>>> I’ve just implemented a new sink, that extends functionality of
>>> existing BucketingSink, currently I’m trying to test functionality,
>>> that is related with timing.
>>> My sink implements *ProcessingTimeCallback, *similarly with the
>>> original BucketingSink. I’m trying to inject
>>> *TestProcessingTimeService *to test
>>>
>>> I discovered, that original tests using some kind of test templates
>>> for testing functions *OneInputStreamOperatorTestHarness*, extending
>>> which test impl of *ProcessingTimeService* is injected,
>>> but this templates are in test scope of *flink-streaming-java
>>> *module, so it could’t be accessed by the external projects.
>>>
>>> May be someone could help me and explain the better approach for
>>> testing sink functionality ?
>>>
>>> Thx !
>>>
>>>
>>
>
Re: How to test new sink
Posted by Rinat <r....@cleverdata.ru>.
Timo, thx for your reply.
I’m using gradle instead of maven, but I’ll look through the existing similar plugins for it.
I don’t think, that sharing of external tests between other projects is a good idea, but it’s out of scope of current discussion.
The main purpose of my request is to understand the existing best practices for testing functions, that uses TimeService and after this, if it’ll be necessary, create an issue with proposals.
Thx.
> On 23 Oct 2017, at 17:51, Timo Walther <tw...@apache.org> wrote:
>
> Hi Rinat,
>
> using one of the Flink test utilities is a good approach to test your custom operators. But of course these classes might change in the future.
>
> First of all, Flink is a open source project so you can just copy the required classes. However, it should be possible to use the Flink test resources like explained here [1]. Flink adds the `test-jar` goal.
>
> @Gary: You are working on the BucketingSink right now, right? Do you have a suggestion?
>
> Regards,
> Timo
>
>
> [1] https://stackoverflow.com/questions/29653914/how-can-i-reference-unit-test-classes-of-a-maven-dependency-in-my-java-project <https://stackoverflow.com/questions/29653914/how-can-i-reference-unit-test-classes-of-a-maven-dependency-in-my-java-project>
>
>
> Am 10/23/17 um 3:16 PM schrieb Rinat:
>> Hi !!!
>>
>> I’ve just implemented a new sink, that extends functionality of existing BucketingSink, currently I’m trying to test functionality, that is related with timing.
>> My sink implements ProcessingTimeCallback, similarly with the original BucketingSink. I’m trying to inject TestProcessingTimeService to test
>>
>> I discovered, that original tests using some kind of test templates for testing functions OneInputStreamOperatorTestHarness, extending which test impl of ProcessingTimeService is injected,
>> but this templates are in test scope of flink-streaming-java module, so it could’t be accessed by the external projects.
>>
>> May be someone could help me and explain the better approach for testing sink functionality ?
>>
>> Thx !
>>
>>
>
Re: How to test new sink
Posted by Timo Walther <tw...@apache.org>.
Hi Rinat,
using one of the Flink test utilities is a good approach to test your
custom operators. But of course these classes might change in the future.
First of all, Flink is a open source project so you can just copy the
required classes. However, it should be possible to use the Flink test
resources like explained here [1]. Flink adds the `test-jar` goal.
@Gary: You are working on the BucketingSink right now, right? Do you
have a suggestion?
Regards,
Timo
[1]
https://stackoverflow.com/questions/29653914/how-can-i-reference-unit-test-classes-of-a-maven-dependency-in-my-java-project
Am 10/23/17 um 3:16 PM schrieb Rinat:
> Hi !!!
>
> I’ve just implemented a new sink, that extends functionality of
> existing BucketingSink, currently I’m trying to test functionality,
> that is related with timing.
> My sink implements *ProcessingTimeCallback, *similarly with the
> original BucketingSink. I’m trying to inject
> *TestProcessingTimeService *to test
>
> I discovered, that original tests using some kind of test templates
> for testing functions *OneInputStreamOperatorTestHarness*, extending
> which test impl of *ProcessingTimeService* is injected,
> but this templates are in test scope of *flink-streaming-java *module,
> so it could’t be accessed by the external projects.
>
> May be someone could help me and explain the better approach for
> testing sink functionality ?
>
> Thx !
>
>