You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by jeffrey <je...@hotmail.com> on 2015/06/16 21:18:54 UTC
unable to reconnect after exception "Channel was inactive for too
long"
In my application, I am running a 5.9 broker locally and the only transport
connector is 2-way SSL.
One endpoint is java, using ActiveMq 5.9, the other is C++ using ActiveMQCPP
3.8.2
After some unspecified time, the CPP application gets an exception that
"Channel was inactive for too long." I attempt to close the consumer(s),
producers(s), session(s), connection(s). Then I create a new connection
factory and get a connection from it.
As soon as I do connection->start(), I get the same exception back again and
again, even though this should be an entirely new connection. I think at
the same time the broker constantly logs one of two things:
2015-06-12 12:27:09,565 | ERROR | Could not accept connection from
tcp://IPADDRESS:58426: javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path validation failed:
java.security.cert.CertPathValidatorException: signature check failed |
org.apache.activemq.broker.TransportConnector | ActiveMQ
BrokerService[broker] Task-6952
or:
2015-06-12 16:45:13,361 | WARN | Transport Connection to:
tcp://IPADDRESS:57964 failed: java.io.EOFException |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
Transport: ssl:///IPADDRESS:57964
I know this is related to inactivity monitor thread, but I do not want to
disable it.
Is there something obvious that I am doing wrong?
Thanks.
--
View this message in context: http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: unable to reconnect after exception "Channel was inactive for too long"
Posted by Tim Bain <tb...@alumni.duke.edu>.
Jeffrey,
In an earlier email, you wrote, "I'm not sure if the Java application ever
gets the exception, but if it is, then it's reconnecting quickly." That
sounds to me like the problem is *not* happening for Java clients, yet your
latest email says "Also, I beleive this happens for both C++ and for Java
clients." Can you help reconcile those statements? If it is happening for
Java clients, please describe what happens, give stack traces, etc.
If it's happening for Java clients too, that may make debugging this
easier, since more people on this list use the Java clients and so more
people might be able to offer insight or try to reproduce it. So that
would be good to know.
Tim
On Wed, Jul 15, 2015 at 8:06 AM, jeffrey <je...@hotmail.com> wrote:
> The keystore and truststore are set in my init method right after the
> activemq::library::ActiveMQCPP::initializeLibrary();
>
> This works, sending and receiving for days or weeks before any issue.
>
> Where there is an issue, I recreate the ConnectionFactory, Connection,
> Session, Consumer, Producer, and then Connection->start() immediately
> throws
> the same issue.
>
> Also, I beleive this happens for both C++ and for Java clients, but not
> necessarilly at the same time.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843p4699338.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: unable to reconnect after exception "Channel was inactive for
too long"
Posted by jeffrey <je...@hotmail.com>.
The keystore and truststore are set in my init method right after the
activemq::library::ActiveMQCPP::initializeLibrary();
This works, sending and receiving for days or weeks before any issue.
Where there is an issue, I recreate the ConnectionFactory, Connection,
Session, Consumer, Producer, and then Connection->start() immediately throws
the same issue.
Also, I beleive this happens for both C++ and for Java clients, but not
necessarilly at the same time.
--
View this message in context: http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843p4699338.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: unable to reconnect after exception "Channel was inactive for too long"
Posted by Tim Bain <tb...@alumni.duke.edu>.
Jeff, does your keystore work fine up until the error happens and then
begin failing (semi-)consistently, or does it fail with this SSL error
right off the bat?
On Tue, Jul 14, 2015 at 7:06 PM, Hadrian Zbarcea <hz...@gmail.com> wrote:
> Jeff,
>
> I would start with figuring out the reason for "PKIX path validation
> failed/signature check failed". It looks like the keystore is not properly
> configured when attempting to reconnect. I'll try to spend some time on
> this tomorrow.
>
> Cheers,
> Hadrian
>
>
>
> On 06/16/2015 03:18 PM, jeffrey wrote:
>
>> In my application, I am running a 5.9 broker locally and the only
>> transport
>> connector is 2-way SSL.
>> One endpoint is java, using ActiveMq 5.9, the other is C++ using
>> ActiveMQCPP
>> 3.8.2
>>
>> After some unspecified time, the CPP application gets an exception that
>> "Channel was inactive for too long." I attempt to close the consumer(s),
>> producers(s), session(s), connection(s). Then I create a new connection
>> factory and get a connection from it.
>>
>> As soon as I do connection->start(), I get the same exception back again
>> and
>> again, even though this should be an entirely new connection. I think at
>> the same time the broker constantly logs one of two things:
>>
>> 2015-06-12 12:27:09,565 | ERROR | Could not accept connection from
>> tcp://IPADDRESS:58426: javax.net.ssl.SSLHandshakeException:
>> sun.security.validator.ValidatorException: PKIX path validation failed:
>> java.security.cert.CertPathValidatorException: signature check failed |
>> org.apache.activemq.broker.TransportConnector | ActiveMQ
>> BrokerService[broker] Task-6952
>>
>> or:
>>
>> 2015-06-12 16:45:13,361 | WARN | Transport Connection to:
>> tcp://IPADDRESS:57964 failed: java.io.EOFException |
>> org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
>> Transport: ssl:///IPADDRESS:57964
>>
>> I know this is related to inactivity monitor thread, but I do not want to
>> disable it.
>> Is there something obvious that I am doing wrong?
>>
>> Thanks.
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
Re: unable to reconnect after exception "Channel was inactive for
too long"
Posted by Hadrian Zbarcea <hz...@gmail.com>.
Jeff,
I would start with figuring out the reason for "PKIX path validation
failed/signature check failed". It looks like the keystore is not
properly configured when attempting to reconnect. I'll try to spend some
time on this tomorrow.
Cheers,
Hadrian
On 06/16/2015 03:18 PM, jeffrey wrote:
> In my application, I am running a 5.9 broker locally and the only transport
> connector is 2-way SSL.
> One endpoint is java, using ActiveMq 5.9, the other is C++ using ActiveMQCPP
> 3.8.2
>
> After some unspecified time, the CPP application gets an exception that
> "Channel was inactive for too long." I attempt to close the consumer(s),
> producers(s), session(s), connection(s). Then I create a new connection
> factory and get a connection from it.
>
> As soon as I do connection->start(), I get the same exception back again and
> again, even though this should be an entirely new connection. I think at
> the same time the broker constantly logs one of two things:
>
> 2015-06-12 12:27:09,565 | ERROR | Could not accept connection from
> tcp://IPADDRESS:58426: javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path validation failed:
> java.security.cert.CertPathValidatorException: signature check failed |
> org.apache.activemq.broker.TransportConnector | ActiveMQ
> BrokerService[broker] Task-6952
>
> or:
>
> 2015-06-12 16:45:13,361 | WARN | Transport Connection to:
> tcp://IPADDRESS:57964 failed: java.io.EOFException |
> org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
> Transport: ssl:///IPADDRESS:57964
>
> I know this is related to inactivity monitor thread, but I do not want to
> disable it.
> Is there something obvious that I am doing wrong?
>
> Thanks.
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: unable to reconnect after exception "Channel was inactive for too long"
Posted by Tim Bain <tb...@alumni.duke.edu>.
There may very well be a bug in the implementation of the InactivityMonitor
code in ActiveMQCPP compared to the Java implementation; I'd recommend you
submit a bug in JIRA, ideally with the client code that produces this
behavior. This one may be harder for someone to track down, because I
don't know of a way to reliably produce "Channel was inactive for too
long." exceptions. (If someone knows of a good way to do that, I'd love to
know it.)
Once this occurs, if you stop your application (but leave the broker up)
and start a new one, does that eliminate the error? Knowing that will let
us determine whether the bad state is in the client or in the broker.
I can't think of any URI options that would help you work around the issue,
though if the restart I asked about in the previous paragraph results in
correct behavior, you could simply exit when you detect the problem and
have the script that started it recognize that it was dead and start a
fresh one. It's not a great workaround, but it might be better than what
you have right now. Or of course you could turn off inactivity monitoring;
you said you don't want to do that, but it might be better than the other
alternatives, at least till someone figures out what's going on and
implements a fix.
Tim
On Wed, Jun 17, 2015 at 1:25 PM, jeffrey <je...@hotmail.com> wrote:
> Tim,
>
> I'm not sure if the Java application ever gets the exception, but if it is,
> then it's reconnecting quickly.
>
> I don't think my application can really control which port it is getting
> moved to, that would be decided after I connect to the server on 61616 and
> we negotiate to a new port.
>
> Are there any transport connector options on the client or server that
> could
> help me here?
>
> Thanks,
> Jeffrey
>
>
> Tim Bain wrote
> > Do you ever get the channel inactivity exception in the Java client? If
> > so, does it reconnect gracefully?
> >
> > Those error messages make it sound like the TCP connection is being
> reused
> > instead of starting a fresh one. Is there any chance you're reusing the
> > same ephemeral port, where the server wouldn't be able to tell that it
> > wasn't simply additional data on the old connection?
> >
> > Tim
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843p4697923.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: unable to reconnect after exception "Channel was inactive for
too long"
Posted by jeffrey <je...@hotmail.com>.
Tim,
I'm not sure if the Java application ever gets the exception, but if it is,
then it's reconnecting quickly.
I don't think my application can really control which port it is getting
moved to, that would be decided after I connect to the server on 61616 and
we negotiate to a new port.
Are there any transport connector options on the client or server that could
help me here?
Thanks,
Jeffrey
Tim Bain wrote
> Do you ever get the channel inactivity exception in the Java client? If
> so, does it reconnect gracefully?
>
> Those error messages make it sound like the TCP connection is being reused
> instead of starting a fresh one. Is there any chance you're reusing the
> same ephemeral port, where the server wouldn't be able to tell that it
> wasn't simply additional data on the old connection?
>
> Tim
--
View this message in context: http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843p4697923.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: unable to reconnect after exception "Channel was inactive for too long"
Posted by Tim Bain <tb...@alumni.duke.edu>.
Do you ever get the channel inactivity exception in the Java client? If
so, does it reconnect gracefully?
Those error messages make it sound like the TCP connection is being reused
instead of starting a fresh one. Is there any chance you're reusing the
same ephemeral port, where the server wouldn't be able to tell that it
wasn't simply additional data on the old connection?
Tim
On Tue, Jun 16, 2015 at 1:18 PM, jeffrey <je...@hotmail.com> wrote:
> In my application, I am running a 5.9 broker locally and the only transport
> connector is 2-way SSL.
> One endpoint is java, using ActiveMq 5.9, the other is C++ using
> ActiveMQCPP
> 3.8.2
>
> After some unspecified time, the CPP application gets an exception that
> "Channel was inactive for too long." I attempt to close the consumer(s),
> producers(s), session(s), connection(s). Then I create a new connection
> factory and get a connection from it.
>
> As soon as I do connection->start(), I get the same exception back again
> and
> again, even though this should be an entirely new connection. I think at
> the same time the broker constantly logs one of two things:
>
> 2015-06-12 12:27:09,565 | ERROR | Could not accept connection from
> tcp://IPADDRESS:58426: javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path validation failed:
> java.security.cert.CertPathValidatorException: signature check failed |
> org.apache.activemq.broker.TransportConnector | ActiveMQ
> BrokerService[broker] Task-6952
>
> or:
>
> 2015-06-12 16:45:13,361 | WARN | Transport Connection to:
> tcp://IPADDRESS:57964 failed: java.io.EOFException |
> org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
> Transport: ssl:///IPADDRESS:57964
>
> I know this is related to inactivity monitor thread, but I do not want to
> disable it.
> Is there something obvious that I am doing wrong?
>
> Thanks.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>