You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2021/09/04 17:24:01 UTC

[jira] [Commented] (BEAM-12523) Number of connection issue

    [ https://issues.apache.org/jira/browse/BEAM-12523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410013#comment-17410013 ] 

Beam JIRA Bot commented on BEAM-12523:
--------------------------------------

This issue was marked "stale-P2" and has not received a public comment in 14 days. It is now automatically moved to P3. If you are still affected by it, you can comment and move it back to P2.

> Number of connection issue
> --------------------------
>
>                 Key: BEAM-12523
>                 URL: https://issues.apache.org/jira/browse/BEAM-12523
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-jms
>    Affects Versions: 2.30.0
>            Reporter: Vincent BALLADA
>            Priority: P3
>
> The maximum number of connections using JmsIO.Read seems to grow in an uncontrolled way, causing new connections to be rejected by the broker if the number exceeds the configured quota on broker side.
> The number of connections is related to the number of workers allocated by the runner, and the number of threads configured per worker, at least at the beginning.
> Our assumption is:
> JmsCheckPointMark is mutable (private Instant oldestMessageTimestamp – this field is updated for each message received). What we think it’s happening, with direct runner at least, the runner is unable to retrieve existing JMSIO readers (JmsCheckPointMark is used as a key - as part of  splitable pardo restriction object - for retrieving active JMSIO Readers, and because it’s mutated for each received message,  the runner will create a new reader for each new received message – that’s why the connections number spins out of control at least in DirectRunner). This is the reason that target parallelism parameter helps on controlling the ‘initial’ number of active connections, but later on we have issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)