Higher level device API for drivers - stepping above a plc field

[Łukasz Dywicki <lu...@code-house.org>]

Hello,
I been thinking of issues I so far found within plc4x api and one of 
them is field-centric approach which bounds client to think in the same 
way. Yet, some protocols think in different categories and there 
field-centric work turns out into a nightmare.

For example within CANopen it is possible to read an variable length 
array of values through SDO transfer (ie. object 0x5000). Yet, in order 
to do that using plc4x apis it is necessary to construct a first call 
(0x5000/0x0) which reads array length and then goes over sub-indexes 
(0x5000,0x1..N). This is not a big burden, but as soon as same structure 
is used across multiple places amount of code to orchestrate array 
retrieval grows. More over, I think that everyone who will get into 
CANopen and SDO read outs will sooner or later copy this logic to his code.
I thought if driver could get it better, but I don't think it would 
without .. object dictionary which is specific to CANopen. Not sure of 
other protocols, as they seem to be rather fine with field level access, 
but I don't think that this measure fits all is right. Looking at more 
complex protocols such as BACnet it is indeed possible to make a client 
which rely on fields, but it will turn caller code into a mess forcing 
further structuring of layers above plc4x just to reflect protocol 
philosophy. On other hand we have a M-Bus which in most of the cases 
works by returning all device measurements at one shot breaking the 1:1 
mapping between request and reply item. We haven't made a consensus how 
it could be addressed, but to me this protocol (as well as WM-Bus) is 
not well fit to keep above contract.

Because of rather bad code I made in 2020 while working on first CANopen 
binding for openHAB I started to re-structure it and improve its 
internal design in 2021. A bit of time have passed and I am again in a 
place where I don't think my work back then was going into right 
direction. First of all, I extracted CoNode, CoConnection and several 
more interfaces, but still at interaction level it does not involve 
object dictionary describing device abilities which is crucial in quick 
bootstrapping of integration. As I am preparing myself to another 
attempt to untie connectorio canopen binding code from specific vendor 
flavor I had to implement, I am back to the problem of 1) where to put 
higher level API 2) how to structure it.
While I currently do not running bacnet not profinet driver from plc4x I 
do smell very similar troubles there, especially with bacnet objects 
(ie. schedules) and profinet configs which reflect GSDML and other 
descriptors out there. Obviously we can always make everything a field, 
but will it make a sense?

_just an idea (tm)_
My brief thinking is about possibility to define an 
"ExtendedDriver<Rq,Rp>" interface (rq - request container, rp - response 
container), which would provide us a generic purpose request/response 
logic as well as subscription stuff. I am sure it will be a bunch of 
work, however in some ways this might feel easier to implement than 
mapping everything to fields and back. This could be compared to a 
stored procedure call within JDBC api which is usually vendor specific. 
Caller of such procedure usually has to handle result according to 
database provider.
In our case through definition of request/response container we could at 
least enforce some basic contract between various equipment.
It would let us also to define a higher level types (subset of PlcObject 
convertible to a PlcStruc's?) to interact:

ExtendedDriver<CANopenReqest, CANopenResponse> extendedDriver;
// assuming CompletableFuture as outcome
// maybe request could optionally have a SDO entry mapper?
extendedDriver.read(new CANopenSDORequest(0x5000, CANopenType.ARRAY))
   .whenComplete((CANopenResponse r, Throwablee ) -> {
     // CANopen specific logic, but well, it is interacting with CANopen
     // and this code knows what to do with it
     MappedData[] data = r.getArray(/*maybe SDO mapper here?*/);
     // CANopenObject o = r.getPlcObject() ??
   })

I don't really have any time frame nor pursue to work on that topic, 
however I wanted to bring it up as I feel we could improve (simplify 
till some degree) device interaction in some cases. Maybe not all, but 
these which are more "object oriented" for sure. I want also to give 
myself enough of time to think of how to make a first attempt without 
making it too dumb nor too complicated.
Above, I think, could also be used for discovery of symbols or objects 
through connections which rely on the same transport as driver itself. 
For example Beckhoff symbols are just read from index within TCP 
connection we make to PLC. Also, if we look at udp discovery in BACnet, 
its just one of operations defined in protocol. Both give us different 
outcome, yet they do work over connection we already made. Obviously it 
can be mapped through connection metadata. While it would fit into this 
API (which is not used anywhere I guess), this approach also gives us a 
trouble of mapping all into plc fields. With an explicit contract of 
field retrieval we are forced to serialize everything to a field, even 
if it does not reflect how given automation system work. Also our field 
definitions will get more and more complicated and harder to keep in 
sync between languages.

Looking forward to your comments on above or counter ideas. Till summer 
season I suppose. :-)

Best,
Łukasz

Re: Higher level device API for drivers - stepping above a plc field

[Sebastian Rühl <sr...@apache.org>]

Hi Łukasz,

at the moment I use the model classes directly but regarding BACNet I also expect the troubles you are describing. So I like the idea to have a way to work with protocol native features directly like you described with the ExtendedDriver.

- Sebastian

On 2022/05/16 20:50:41 Łukasz Dywicki wrote:
> Hello,
> I been thinking of issues I so far found within plc4x api and one of 
> them is field-centric approach which bounds client to think in the same 
> way. Yet, some protocols think in different categories and there 
> field-centric work turns out into a nightmare.
> 
> For example within CANopen it is possible to read an variable length 
> array of values through SDO transfer (ie. object 0x5000). Yet, in order 
> to do that using plc4x apis it is necessary to construct a first call 
> (0x5000/0x0) which reads array length and then goes over sub-indexes 
> (0x5000,0x1..N). This is not a big burden, but as soon as same structure 
> is used across multiple places amount of code to orchestrate array 
> retrieval grows. More over, I think that everyone who will get into 
> CANopen and SDO read outs will sooner or later copy this logic to his code.
> I thought if driver could get it better, but I don't think it would 
> without .. object dictionary which is specific to CANopen. Not sure of 
> other protocols, as they seem to be rather fine with field level access, 
> but I don't think that this measure fits all is right. Looking at more 
> complex protocols such as BACnet it is indeed possible to make a client 
> which rely on fields, but it will turn caller code into a mess forcing 
> further structuring of layers above plc4x just to reflect protocol 
> philosophy. On other hand we have a M-Bus which in most of the cases 
> works by returning all device measurements at one shot breaking the 1:1 
> mapping between request and reply item. We haven't made a consensus how 
> it could be addressed, but to me this protocol (as well as WM-Bus) is 
> not well fit to keep above contract.
> 
> Because of rather bad code I made in 2020 while working on first CANopen 
> binding for openHAB I started to re-structure it and improve its 
> internal design in 2021. A bit of time have passed and I am again in a 
> place where I don't think my work back then was going into right 
> direction. First of all, I extracted CoNode, CoConnection and several 
> more interfaces, but still at interaction level it does not involve 
> object dictionary describing device abilities which is crucial in quick 
> bootstrapping of integration. As I am preparing myself to another 
> attempt to untie connectorio canopen binding code from specific vendor 
> flavor I had to implement, I am back to the problem of 1) where to put 
> higher level API 2) how to structure it.
> While I currently do not running bacnet not profinet driver from plc4x I 
> do smell very similar troubles there, especially with bacnet objects 
> (ie. schedules) and profinet configs which reflect GSDML and other 
> descriptors out there. Obviously we can always make everything a field, 
> but will it make a sense?
> 
> _just an idea (tm)_
> My brief thinking is about possibility to define an 
> "ExtendedDriver<Rq,Rp>" interface (rq - request container, rp - response 
> container), which would provide us a generic purpose request/response 
> logic as well as subscription stuff. I am sure it will be a bunch of 
> work, however in some ways this might feel easier to implement than 
> mapping everything to fields and back. This could be compared to a 
> stored procedure call within JDBC api which is usually vendor specific. 
> Caller of such procedure usually has to handle result according to 
> database provider.
> In our case through definition of request/response container we could at 
> least enforce some basic contract between various equipment.
> It would let us also to define a higher level types (subset of PlcObject 
> convertible to a PlcStruc's?) to interact:
> 
> ExtendedDriver<CANopenReqest, CANopenResponse> extendedDriver;
> // assuming CompletableFuture as outcome
> // maybe request could optionally have a SDO entry mapper?
> extendedDriver.read(new CANopenSDORequest(0x5000, CANopenType.ARRAY))
>    .whenComplete((CANopenResponse r, Throwablee ) -> {
>      // CANopen specific logic, but well, it is interacting with CANopen
>      // and this code knows what to do with it
>      MappedData[] data = r.getArray(/*maybe SDO mapper here?*/);
>      // CANopenObject o = r.getPlcObject() ??
>    })
> 
> I don't really have any time frame nor pursue to work on that topic, 
> however I wanted to bring it up as I feel we could improve (simplify 
> till some degree) device interaction in some cases. Maybe not all, but 
> these which are more "object oriented" for sure. I want also to give 
> myself enough of time to think of how to make a first attempt without 
> making it too dumb nor too complicated.
> Above, I think, could also be used for discovery of symbols or objects 
> through connections which rely on the same transport as driver itself. 
> For example Beckhoff symbols are just read from index within TCP 
> connection we make to PLC. Also, if we look at udp discovery in BACnet, 
> its just one of operations defined in protocol. Both give us different 
> outcome, yet they do work over connection we already made. Obviously it 
> can be mapped through connection metadata. While it would fit into this 
> API (which is not used anywhere I guess), this approach also gives us a 
> trouble of mapping all into plc fields. With an explicit contract of 
> field retrieval we are forced to serialize everything to a field, even 
> if it does not reflect how given automation system work. Also our field 
> definitions will get more and more complicated and harder to keep in 
> sync between languages.
> 
> Looking forward to your comments on above or counter ideas. Till summer 
> season I suppose. :-)
> 
> Best,
> Łukasz
>