You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@plc4x.apache.org by Christofer Dutz <ch...@c-ware.de> on 2019/04/10 08:32:31 UTC

Anyone interested in revamping the Kafka-Connect adapter?

Hi all,

yesterday I have been trying to understand the Kafka-Connect adapter and think I managed to get to a point where I sort of understand it.
However I’m not quite happy with the way it currently works (I think you have to put every item address in a space-separate list also does it use manual polling)

I know Kafka-Connect supports both push and pull types … I think sort of using the new Scraper and it’s trigger strategies makes it a perfect candidate
for changing the connector into a pull-connector. Also do I think we have to provide an alternate way of configuring … I know that one poc of collecting
2600 Items on 200 PLCs was quite a long configuration string ;-)

Anyone interested in taking on the fight? Ideally one with either Scraper knowledge and/or Kafka … I sort of don’t qualify for either of them ;-)

Chris

Re: Anyone interested in revamping the Kafka-Connect adapter?

Posted by Tim Mitsch <t....@pragmaticindustries.de>.
Hey

I'm not so deep in Kafka-Connect - but i would be interested to work on the camel implementation of scraper - i will do a jira issue for that, good point Julian.
But time is quite rare until weekend.

Best
Tim

Am 10.04.19, 10:35 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:

    Hi,
    
    yes, that should be the way to go (also for something like Camel...).
    I have no time to contriobute this week, I can only offer to answer questions... perhaps Tim can say something?
    
    Julian
    
    PS.: Oh, we should at least add Jira issues... we have to learn that
    
    Am 10.04.19, 10:32 schrieb "Christofer Dutz" <ch...@c-ware.de>:
    
        Hi all,
        
        yesterday I have been trying to understand the Kafka-Connect adapter and think I managed to get to a point where I sort of understand it.
        However I’m not quite happy with the way it currently works (I think you have to put every item address in a space-separate list also does it use manual polling)
        
        I know Kafka-Connect supports both push and pull types … I think sort of using the new Scraper and it’s trigger strategies makes it a perfect candidate
        for changing the connector into a pull-connector. Also do I think we have to provide an alternate way of configuring … I know that one poc of collecting
        2600 Items on 200 PLCs was quite a long configuration string ;-)
        
        Anyone interested in taking on the fight? Ideally one with either Scraper knowledge and/or Kafka … I sort of don’t qualify for either of them ;-)
        
        Chris

Re: Anyone interested in revamping the Kafka-Connect adapter?

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi,

yes, that should be the way to go (also for something like Camel...).
I have no time to contriobute this week, I can only offer to answer questions... perhaps Tim can say something?

Julian

PS.: Oh, we should at least add Jira issues... we have to learn that

Am 10.04.19, 10:32 schrieb "Christofer Dutz" <ch...@c-ware.de>:

    Hi all,
    
    yesterday I have been trying to understand the Kafka-Connect adapter and think I managed to get to a point where I sort of understand it.
    However I’m not quite happy with the way it currently works (I think you have to put every item address in a space-separate list also does it use manual polling)
    
    I know Kafka-Connect supports both push and pull types … I think sort of using the new Scraper and it’s trigger strategies makes it a perfect candidate
    for changing the connector into a pull-connector. Also do I think we have to provide an alternate way of configuring … I know that one poc of collecting
    2600 Items on 200 PLCs was quite a long configuration string ;-)
    
    Anyone interested in taking on the fight? Ideally one with either Scraper knowledge and/or Kafka … I sort of don’t qualify for either of them ;-)
    
    Chris