You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@plc4x.apache.org by Christofer Dutz <ch...@c-ware.de> on 2018/11/28 15:28:34 UTC

[All drivers] Noticed a problem in the way we handle responses

Hi all,

today we started working on a first codecentric Plc4x POC using the Siemens S7 driver.
Unfortunately we stumbled into a problem that will probably affect other drivers too.
In the IsoOnTcpProtocol, I check if the packet starts with the IsoOnTcp magic number and if enough bytes have been read.
I was expecting Netty to re-send the buffer, if I don’t consume it. This worked nicely in cases of slow networks. However in case of a fast network, Netty seems to chunk the responses into 512 byte chunks.
If I don’t process these, they are not re-sent. Then the next call comes and it doesn’t detect the magic byte at the beginning and then an exception is thrown.

I need to work out how to handle this case (Fast network and responses with a TCP payload greater than 512 bytes)

I’m tracking things in
PLC4X-76 - When receiving responses with more than 512 byte, the IsoOnTcp protocol doesn't work
https://issues.apache.org/jira/browse/PLC4X-76

Think we should double check the other drivers as I’m expecting Netty to behave similarly in other drivers too.

Chris

AW: [All drivers] Noticed a problem in the way we handle responses

Posted by Christofer Dutz <ch...@c-ware.de>.
Hi Julian,

I guess this was related exactly to this case ... luckily the Netty guys at StackOverflow were able to help quickly.
It turned out that in the end I had to change the protocol layer that sits on top of TCP from MessageToMessageCodec to ByteToMessageCodec. 
My general algorithm checking the number of bytes and just returning in case of not-enough bytes was correct, I just had to switch the parent class.

So we should ensure the lowest level protocol always uses ByteToMessageCodec

But now I was able to request 2640 individual addresses from a S7 1500 in 190-210ms.

For our current POC, the limit of the existing system was about 2 seconds for the same amount, the goal for the POC was 500ms ... 
so guess I have to introduce Thread.sleeps into the code to meet the expectations __

This is correctly split up into 33 sub-messages with each about 79 addresses.

... works like a charm.

https://sparrowmissions.com/wp-content/uploads/2016/05/The-A-Team.jpg

Chris

Am 29.11.18, 09:59 schrieb "Julian Feinauer" <j....@pragmaticminds.de>:

    Hey Chris,
    
    I', not to deep into this, but I observed a "wrong magic byte..." or so error message from time to time last week.
    This came while working at the plc scraper when sending many requests (and cancling long running ones).
    Could this be a related problem?
    
    Julian
    
    Am 28.11.18, 16:28 schrieb "Christofer Dutz" <ch...@c-ware.de>:
    
        Hi all,
        
        today we started working on a first codecentric Plc4x POC using the Siemens S7 driver.
        Unfortunately we stumbled into a problem that will probably affect other drivers too.
        In the IsoOnTcpProtocol, I check if the packet starts with the IsoOnTcp magic number and if enough bytes have been read.
        I was expecting Netty to re-send the buffer, if I don’t consume it. This worked nicely in cases of slow networks. However in case of a fast network, Netty seems to chunk the responses into 512 byte chunks.
        If I don’t process these, they are not re-sent. Then the next call comes and it doesn’t detect the magic byte at the beginning and then an exception is thrown.
        
        I need to work out how to handle this case (Fast network and responses with a TCP payload greater than 512 bytes)
        
        I’m tracking things in
        PLC4X-76 - When receiving responses with more than 512 byte, the IsoOnTcp protocol doesn't work
        https://issues.apache.org/jira/browse/PLC4X-76
        
        Think we should double check the other drivers as I’m expecting Netty to behave similarly in other drivers too.
        
        Chris

Re: [All drivers] Noticed a problem in the way we handle responses

Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hey Chris,

I', not to deep into this, but I observed a "wrong magic byte..." or so error message from time to time last week.
This came while working at the plc scraper when sending many requests (and cancling long running ones).
Could this be a related problem?

Julian

Am 28.11.18, 16:28 schrieb "Christofer Dutz" <ch...@c-ware.de>:

    Hi all,
    
    today we started working on a first codecentric Plc4x POC using the Siemens S7 driver.
    Unfortunately we stumbled into a problem that will probably affect other drivers too.
    In the IsoOnTcpProtocol, I check if the packet starts with the IsoOnTcp magic number and if enough bytes have been read.
    I was expecting Netty to re-send the buffer, if I don’t consume it. This worked nicely in cases of slow networks. However in case of a fast network, Netty seems to chunk the responses into 512 byte chunks.
    If I don’t process these, they are not re-sent. Then the next call comes and it doesn’t detect the magic byte at the beginning and then an exception is thrown.
    
    I need to work out how to handle this case (Fast network and responses with a TCP payload greater than 512 bytes)
    
    I’m tracking things in
    PLC4X-76 - When receiving responses with more than 512 byte, the IsoOnTcp protocol doesn't work
    https://issues.apache.org/jira/browse/PLC4X-76
    
    Think we should double check the other drivers as I’m expecting Netty to behave similarly in other drivers too.
    
    Chris