You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dipanjan Mazumder <ja...@yahoo.com> on 2021/05/25 07:26:34 UTC

[Flink-Siddhi]: Issue processing siddhi CEP task on flink

 
Hi Guys,
     I am facing another integration challenge with flink-siddhi which i am not able to crack yet , i am trying to debug the flow with flink but its too much and taking a lot of time , being new to flink. My problem is basically when i publish message through kafka and the flink aplication/job has CEP processing operation using SiddiCEP , taskmanager is not processing the task with the operator having siddhi CEP. It is not even throwing any error anywhere. So i started debugging the taskmanager but i am not able to reach anywhere yet have been doing that from last 4 days with no luck. I have created a stackoverflow query which has more details on the program and application. 
Flink version : 1.13Scala: 2.11Java: 1.8I am using flink-siddhi library: haoch/flink-siddhi 2.11-0.2.2-snapshot version

| 
| 
| 
|  |  |

 |

 |
| 
|  | 
haoch/flink-siddhi

A CEP library to run Siddhi within Apache Flink™ Streaming Application - haoch/flink-siddhi
 |

 |

 |




The link is :    Not able to process kafka json message with Flink siddhi library

| 
| 
| 
|  |  |

 |

 |
| 
|  | 
Not able to process kafka json message with Flink siddhi library

I am trying to create a simple application where the app will consume Kafka message do some cql transform and pu...
 |

 |

 |





If there is any pointer that can be given on the problem , it would save my day and life. So please kindly respond on the same.
Thanks in advance.
RegardsDipanjan    On Tuesday, May 25, 2021, 12:48:59 PM GMT+5:30, Dipanjan Mazumder <ja...@yahoo.com> wrote:  
 
  Hi All,
    Found the solution , 
Problem: I was actually using an intermediate library to integrate siddhi with Flink (https://github.com/haoch/flink-siddhi) and i was creating a SiddhiCEP instance and then calling "define()" on that instance , while i was registering the extension on the created instance , but the define method was creating an internal SiddhiCEP instance and using that for processing. So i found that out by debugging the application. It is an implementation problem with the library itself.
Solution: i used from() method on the created SiddhiCEP instance instead of define() and it was using the CEP i created for doing the rest of processing and the registered extension were accounted similarly and so they were recognised at runtime.
RegardsDipanjan
    On Friday, May 21, 2021, 01:51:09 PM GMT+5:30, Salva Alcántara <sa...@gmail.com> wrote:  
 
 Hi Dipanjan,

I agree with Till. If the extensions are are included in the jar for your
job, it should work. I was having the same doubts some weeks a go and can
confirm that as long as the jar includes those extensions, it works.

One thing I needed to do is to register the different extensions. For
example:

```
siddhiManager.setExtension("map:create", classOf[CreateFunctionExtension])
```

Regards,

Salva



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/