You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@plc4x.apache.org by Julian Feinauer <j....@pragmaticminds.de> on 2019/02/06 09:37:27 UTC

Move plc4x integrations to Apache Edgent (indubating)

Hi all,

first, I decided to cross post, as this addresses both projects equally, PLC4X and Edgent.
Second, this post is mainly about bringing my thoughts and chat with Chris to the list(s).

As we have the idea to “revive” Apache Edgent as Edge Framework we think that it would be better in the long run to move the PLC4X integrations to Apache Edgent.
This means, that we would build a good PLC4X “Source” for Apache Edgent and then move all integrations as “Sinks” for Apache Edgent.
Then, one could leverage all of Apache Edgents Stream Processing features on PLC data and even join it with non-PLC Data and send it to futher downstream systems.
Another goal is to integrate our CRUNCH Framework [1] into Edgent to allow analysis of these Streams in Edgent directly.

What are your thoughts on that?
Julian

[1] https://github.com/pragmaticminds/crunch

Re: Move plc4x integrations to Apache Edgent (indubating)

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi Chris,

again cross-posting, but for the last time.
I write this now from an edgent perspective (also being PPMC there).

Is there any objection against moving some integrations from PLC4X to edgent and play around with that?
So we would have duplicate code for a short time and if it works all in edgent I would remove it from the PLC4X codebase.
I think we can simply copy it over as long as we state that it is copied from PLC4X, or?

Especially I think about all the things that are going on around the scraper which could be extended in edgent.

Oh, and I suggest to keep this discussion now on the edgent list.

Julian

Am 06.02.19, 10:48 schrieb "Christofer Dutz" <ch...@c-ware.de>:

    Hi all,
    
    also cross-replying ... I know my mail client is going to hate me for it ;-)
    
    Well PLC4X would not only provide Sinks, but also Sources ... and I agree that having the code in Edgent makes far more sense. In general I think almost all integration modules make sense to be moved to the projects they integrate into.
    As I mentioned in that chat you are referring to, the reason for most integrations being in our repo is simply that this way we were able to write, commit and release code, as we are usually not committers in most of the projects we're writing integrations for.
    
    From a maintenance perspective this also has big benefits as - at least speaking for myself - I'm usually not that deeply involved in the project 
    I integrate and having the integrations on the other side could ensure maintenance by people that actually know what they're doing ;-)
    
    I would suggest we contact the incubating projects we currently integrate into (currently only Edgent) now ... for others (outside of the ASF and TLPs), I would suggest we wait until PLC4X is a TLP and then contact them (Camel, Calcite, Kafka, ...)
    
    But I'd like to express my big +1 to this proposal (Being a member of both projects PPMCs)
    
    Chris
    
    
    
    Am 06.02.19, 10:37 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:
    
        Hi all,
        
        first, I decided to cross post, as this addresses both projects equally, PLC4X and Edgent.
        Second, this post is mainly about bringing my thoughts and chat with Chris to the list(s).
        
        As we have the idea to “revive” Apache Edgent as Edge Framework we think that it would be better in the long run to move the PLC4X integrations to Apache Edgent.
        This means, that we would build a good PLC4X “Source” for Apache Edgent and then move all integrations as “Sinks” for Apache Edgent.
        Then, one could leverage all of Apache Edgents Stream Processing features on PLC data and even join it with non-PLC Data and send it to futher downstream systems.
        Another goal is to integrate our CRUNCH Framework [1] into Edgent to allow analysis of these Streams in Edgent directly.
        
        What are your thoughts on that?
        Julian
        
        [1] https://github.com/pragmaticminds/crunch

Re: Move plc4x integrations to Apache Edgent (indubating)

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi Chris,

again cross-posting, but for the last time.
I write this now from an edgent perspective (also being PPMC there).

Is there any objection against moving some integrations from PLC4X to edgent and play around with that?
So we would have duplicate code for a short time and if it works all in edgent I would remove it from the PLC4X codebase.
I think we can simply copy it over as long as we state that it is copied from PLC4X, or?

Especially I think about all the things that are going on around the scraper which could be extended in edgent.

Oh, and I suggest to keep this discussion now on the edgent list.

Julian

Am 06.02.19, 10:48 schrieb "Christofer Dutz" <ch...@c-ware.de>:

    Hi all,
    
    also cross-replying ... I know my mail client is going to hate me for it ;-)
    
    Well PLC4X would not only provide Sinks, but also Sources ... and I agree that having the code in Edgent makes far more sense. In general I think almost all integration modules make sense to be moved to the projects they integrate into.
    As I mentioned in that chat you are referring to, the reason for most integrations being in our repo is simply that this way we were able to write, commit and release code, as we are usually not committers in most of the projects we're writing integrations for.
    
    From a maintenance perspective this also has big benefits as - at least speaking for myself - I'm usually not that deeply involved in the project 
    I integrate and having the integrations on the other side could ensure maintenance by people that actually know what they're doing ;-)
    
    I would suggest we contact the incubating projects we currently integrate into (currently only Edgent) now ... for others (outside of the ASF and TLPs), I would suggest we wait until PLC4X is a TLP and then contact them (Camel, Calcite, Kafka, ...)
    
    But I'd like to express my big +1 to this proposal (Being a member of both projects PPMCs)
    
    Chris
    
    
    
    Am 06.02.19, 10:37 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:
    
        Hi all,
        
        first, I decided to cross post, as this addresses both projects equally, PLC4X and Edgent.
        Second, this post is mainly about bringing my thoughts and chat with Chris to the list(s).
        
        As we have the idea to “revive” Apache Edgent as Edge Framework we think that it would be better in the long run to move the PLC4X integrations to Apache Edgent.
        This means, that we would build a good PLC4X “Source” for Apache Edgent and then move all integrations as “Sinks” for Apache Edgent.
        Then, one could leverage all of Apache Edgents Stream Processing features on PLC data and even join it with non-PLC Data and send it to futher downstream systems.
        Another goal is to integrate our CRUNCH Framework [1] into Edgent to allow analysis of these Streams in Edgent directly.
        
        What are your thoughts on that?
        Julian
        
        [1] https://github.com/pragmaticminds/crunch

Re: Move plc4x integrations to Apache Edgent (indubating)

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi,

yes, clarification... Sherlock Holmes once said that there is nothing more obfuscating than an obvious fact... : )

So, what I think edgent should do is to provide suitable inputs (IoT-y inputs, like PLC4X, OBD2, ...) and provide its outputs that are again IoT / Cloud based.
This would mean that we could or should also consider to move integrations like NiFi or Calcite to Edgent because we then have a resolution of the m-n Problem with n inputs and m outputs.
And we also get OBD2 to Nifi and ... .
Do you get my point?

Julian

Am 06.02.19, 10:55 schrieb "Christofer Dutz" <ch...@c-ware.de>:

    Maybe we should refine the topic ... we discussed only to move only the plc4x-edgent integration to Edgent ... at least it way my perception to move only that ;)
    
    Chris
    
    Am 06.02.19, 10:48 schrieb "Christofer Dutz" <ch...@c-ware.de>:
    
        Hi all,
        
        also cross-replying ... I know my mail client is going to hate me for it ;-)
        
        Well PLC4X would not only provide Sinks, but also Sources ... and I agree that having the code in Edgent makes far more sense. In general I think almost all integration modules make sense to be moved to the projects they integrate into.
        As I mentioned in that chat you are referring to, the reason for most integrations being in our repo is simply that this way we were able to write, commit and release code, as we are usually not committers in most of the projects we're writing integrations for.
        
        From a maintenance perspective this also has big benefits as - at least speaking for myself - I'm usually not that deeply involved in the project 
        I integrate and having the integrations on the other side could ensure maintenance by people that actually know what they're doing ;-)
        
        I would suggest we contact the incubating projects we currently integrate into (currently only Edgent) now ... for others (outside of the ASF and TLPs), I would suggest we wait until PLC4X is a TLP and then contact them (Camel, Calcite, Kafka, ...)
        
        But I'd like to express my big +1 to this proposal (Being a member of both projects PPMCs)
        
        Chris
        
        
        
        Am 06.02.19, 10:37 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:
        
            Hi all,
            
            first, I decided to cross post, as this addresses both projects equally, PLC4X and Edgent.
            Second, this post is mainly about bringing my thoughts and chat with Chris to the list(s).
            
            As we have the idea to “revive” Apache Edgent as Edge Framework we think that it would be better in the long run to move the PLC4X integrations to Apache Edgent.
            This means, that we would build a good PLC4X “Source” for Apache Edgent and then move all integrations as “Sinks” for Apache Edgent.
            Then, one could leverage all of Apache Edgents Stream Processing features on PLC data and even join it with non-PLC Data and send it to futher downstream systems.
            Another goal is to integrate our CRUNCH Framework [1] into Edgent to allow analysis of these Streams in Edgent directly.
            
            What are your thoughts on that?
            Julian
            
            [1] https://github.com/pragmaticminds/crunch

Re: Move plc4x integrations to Apache Edgent (indubating)

Posted by Christofer Dutz <ch...@c-ware.de>.
Maybe we should refine the topic ... we discussed only to move only the plc4x-edgent integration to Edgent ... at least it way my perception to move only that ;)

Chris

Am 06.02.19, 10:48 schrieb "Christofer Dutz" <ch...@c-ware.de>:

    Hi all,
    
    also cross-replying ... I know my mail client is going to hate me for it ;-)
    
    Well PLC4X would not only provide Sinks, but also Sources ... and I agree that having the code in Edgent makes far more sense. In general I think almost all integration modules make sense to be moved to the projects they integrate into.
    As I mentioned in that chat you are referring to, the reason for most integrations being in our repo is simply that this way we were able to write, commit and release code, as we are usually not committers in most of the projects we're writing integrations for.
    
    From a maintenance perspective this also has big benefits as - at least speaking for myself - I'm usually not that deeply involved in the project 
    I integrate and having the integrations on the other side could ensure maintenance by people that actually know what they're doing ;-)
    
    I would suggest we contact the incubating projects we currently integrate into (currently only Edgent) now ... for others (outside of the ASF and TLPs), I would suggest we wait until PLC4X is a TLP and then contact them (Camel, Calcite, Kafka, ...)
    
    But I'd like to express my big +1 to this proposal (Being a member of both projects PPMCs)
    
    Chris
    
    
    
    Am 06.02.19, 10:37 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:
    
        Hi all,
        
        first, I decided to cross post, as this addresses both projects equally, PLC4X and Edgent.
        Second, this post is mainly about bringing my thoughts and chat with Chris to the list(s).
        
        As we have the idea to “revive” Apache Edgent as Edge Framework we think that it would be better in the long run to move the PLC4X integrations to Apache Edgent.
        This means, that we would build a good PLC4X “Source” for Apache Edgent and then move all integrations as “Sinks” for Apache Edgent.
        Then, one could leverage all of Apache Edgents Stream Processing features on PLC data and even join it with non-PLC Data and send it to futher downstream systems.
        Another goal is to integrate our CRUNCH Framework [1] into Edgent to allow analysis of these Streams in Edgent directly.
        
        What are your thoughts on that?
        Julian
        
        [1] https://github.com/pragmaticminds/crunch

Re: Move plc4x integrations to Apache Edgent (indubating)

Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,

also cross-replying ... I know my mail client is going to hate me for it ;-)

Well PLC4X would not only provide Sinks, but also Sources ... and I agree that having the code in Edgent makes far more sense. In general I think almost all integration modules make sense to be moved to the projects they integrate into.
As I mentioned in that chat you are referring to, the reason for most integrations being in our repo is simply that this way we were able to write, commit and release code, as we are usually not committers in most of the projects we're writing integrations for.

From a maintenance perspective this also has big benefits as - at least speaking for myself - I'm usually not that deeply involved in the project 
I integrate and having the integrations on the other side could ensure maintenance by people that actually know what they're doing ;-)

I would suggest we contact the incubating projects we currently integrate into (currently only Edgent) now ... for others (outside of the ASF and TLPs), I would suggest we wait until PLC4X is a TLP and then contact them (Camel, Calcite, Kafka, ...)

But I'd like to express my big +1 to this proposal (Being a member of both projects PPMCs)

Chris



Am 06.02.19, 10:37 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:

    Hi all,
    
    first, I decided to cross post, as this addresses both projects equally, PLC4X and Edgent.
    Second, this post is mainly about bringing my thoughts and chat with Chris to the list(s).
    
    As we have the idea to “revive” Apache Edgent as Edge Framework we think that it would be better in the long run to move the PLC4X integrations to Apache Edgent.
    This means, that we would build a good PLC4X “Source” for Apache Edgent and then move all integrations as “Sinks” for Apache Edgent.
    Then, one could leverage all of Apache Edgents Stream Processing features on PLC data and even join it with non-PLC Data and send it to futher downstream systems.
    Another goal is to integrate our CRUNCH Framework [1] into Edgent to allow analysis of these Streams in Edgent directly.
    
    What are your thoughts on that?
    Julian
    
    [1] https://github.com/pragmaticminds/crunch

Re: Move plc4x integrations to Apache Edgent (indubating)

Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,

also cross-replying ... I know my mail client is going to hate me for it ;-)

Well PLC4X would not only provide Sinks, but also Sources ... and I agree that having the code in Edgent makes far more sense. In general I think almost all integration modules make sense to be moved to the projects they integrate into.
As I mentioned in that chat you are referring to, the reason for most integrations being in our repo is simply that this way we were able to write, commit and release code, as we are usually not committers in most of the projects we're writing integrations for.

From a maintenance perspective this also has big benefits as - at least speaking for myself - I'm usually not that deeply involved in the project 
I integrate and having the integrations on the other side could ensure maintenance by people that actually know what they're doing ;-)

I would suggest we contact the incubating projects we currently integrate into (currently only Edgent) now ... for others (outside of the ASF and TLPs), I would suggest we wait until PLC4X is a TLP and then contact them (Camel, Calcite, Kafka, ...)

But I'd like to express my big +1 to this proposal (Being a member of both projects PPMCs)

Chris



Am 06.02.19, 10:37 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:

    Hi all,
    
    first, I decided to cross post, as this addresses both projects equally, PLC4X and Edgent.
    Second, this post is mainly about bringing my thoughts and chat with Chris to the list(s).
    
    As we have the idea to “revive” Apache Edgent as Edge Framework we think that it would be better in the long run to move the PLC4X integrations to Apache Edgent.
    This means, that we would build a good PLC4X “Source” for Apache Edgent and then move all integrations as “Sinks” for Apache Edgent.
    Then, one could leverage all of Apache Edgents Stream Processing features on PLC data and even join it with non-PLC Data and send it to futher downstream systems.
    Another goal is to integrate our CRUNCH Framework [1] into Edgent to allow analysis of these Streams in Edgent directly.
    
    What are your thoughts on that?
    Julian
    
    [1] https://github.com/pragmaticminds/crunch