[jira] [Comment Edited] (IGNITE-3303) Apache Flink Integration - Flink source to run a continuous query against one or multiple caches

["Anton Vinogradov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657165#comment-15657165 ] 

Anton Vinogradov edited comment on IGNITE-3303 at 11/11/16 2:19 PM:
--------------------------------------------------------------------

Saikat,

I've started review and still see that 
{noformat}private static volatile boolean stopped = true;{noformat}
still static. 
What give us no chance to use more that one {{IgniteSource}}

So I tried to change it to nonstatiс and tests failed.
I've started to check reasons and found that {{IgniteSource}} instance at test and under flink (when flink run IgniteSource.run()) are different.
Just set breakpoint inside {{IgniteSource.start}} and {{IgniteSource.run}} and you'll se that they are different.

Af far as I understand flink makes object copy. 

I've discovered web for some {{RichParallelSourceFunction}} implementations and found they use another strategy to work with 
{noformat}private volatile boolean isRunning = false;{noformat}
for example 
http://www.programcreek.com/java-api-examples/index.php?source_dir=StreamKV-master/streamkv-java/src/main/java/streamkv/api/java/benchmark/AsyncKVLocalBenchmark.java

So, my question is what was the reason to use static {{private static volatile boolean stopped = true;}} and is there any changes to make tests with more than one {{IgniteSource}}?
Also, I see, that parallelism is always 1 at tests. Does it means that implementation support only "1" case?

P.s.  Saikat, I read some articles about Flink last hours, and I'm still not flink guru :) 
Possible I'm thinking in the wrong direction? 





was (Author: avinogradov):
Saikat,

I've started review and still see that 
{noformat}private static volatile boolean stopped = true;{noformat}
still static. 
What give us no chance to use more that one {noformat}IgniteSource{noformat}

So I tried to change it to nonstatiс and tests failed.
I've started to check reasons and found that {noformat}IgniteSource{noformat} instance at test and under flink (when flink run IgniteSource.run()) are different.
Just set breakpoint inside {noformat}IgniteSource.start{noformat} and {noformat}IgniteSource.run{noformat} and you'll se that they are different.

Af far as I understand flink makes object copy. 

I've discovered web for some {noformat}RichParallelSourceFunction{noformat} implementations and found they use another strategy to work with 
{noformat}private volatile boolean isRunning = false;{noformat}
for example 
http://www.programcreek.com/java-api-examples/index.php?source_dir=StreamKV-master/streamkv-java/src/main/java/streamkv/api/java/benchmark/AsyncKVLocalBenchmark.java

So, my question is what was the reason to use static {noformat}private static volatile boolean stopped = true;{noformat} and is there any changes to make tests with more than one {noformat}IgniteSource{noformat}?
Also, I see, that parallelism is always 1 at tests. Does it means that implementation support only "1" case?

P.s.  Saikat, I read some articles about Flink last hours, and I'm still not flink guru :) 
Possible I'm thinking in the wrong direction? 




> Apache Flink Integration - Flink source to run a continuous query against one or multiple caches
> ------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-3303
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3303
>             Project: Ignite
>          Issue Type: New Feature
>          Components: streaming
>            Reporter: Saikat Maitra
>            Assignee: Saikat Maitra
>         Attachments: Screen Shot 2016-10-07 at 12.44.47 AM.png, testFlinkIgniteSourceWithLargeBatch.log, win7.PNG
>
>
> Apache Flink integration 
> +++ *Ignite as a bidirectional Connector* +++
> As a Flink source => run a continuous query against one or multiple
> caches [4].
> Related discussion : http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Flink-lt-gt-Apache-Ignite-integration-td8163.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)