You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Jaroslaw Cwiklik <cw...@apache.org> on 2017/12/14 21:52:45 UTC
[VOTE] UIMA-AS 2.10.2 RC5
Hi,
the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
This version contains the following fixes:
- fixed documentation
The list of changes in Jira:
http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.10.2AS%20AND%20project%20%3D%20UIMA
The source and binary zip/tars are staged to
http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
The eclipse update subsite is here:
http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
The Maven artifacts are here:
http://repository.apache.org/content/repositories/orgapacheuima-1177
The SVN tags are here:
http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
See http://uima.apache.org/testing-builds.html for suggestions on how to
test release candidates.
Please vote on release:
[ ] +1 OK to release
[ ] 0 Don't care
[ ] -1 Not OK to release, because ...
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Burn Lewis <bu...@gmail.com>.
- Checked signatures - OK
- Tested binary with a dozen CASes - OK
- Built from source & checked signatures - OK
- Tested source build with a dozen CASes - OK
- Spot checked README, RELEASE_NOTES, LICENSE, NOTICE, JIRAs, docs - OK
- Tested TargetServiceId - OK
Minor complaint - The (undocumented) -TargetServiceId option on
RunRemoteAsyncAE does not send a targeted CAS.
[ ] +1 OK to release
~Burn
On Tue, Jan 2, 2018 at 11:21 AM, Jaroslaw Cwiklik <cw...@apache.org>
wrote:
> "Many other" = "all services " were kaput? If so, it could be the broker
> failure (what version?) or something else. Sorry cant pinpoint this stuff
> with accuracy since the broken connection can be symptom of many problems.
> Is this system in the kaput state still?
>
> Jerry
>
> On Tue, Jan 2, 2018 at 11:16 AM, Eddie Epstein <ea...@gmail.com>
> wrote:
>
> > There were no long GCs for this service. Note (from above) that many
> other
> > services running older versions of uima-as were also showing the same
> > behavior.
> >
> > Eddie
> >
> > On Tue, Jan 2, 2018 at 9:21 AM, Jaroslaw Cwiklik <cw...@apache.org>
> > wrote:
> >
> > > Looks like the client lost connection to the broker for some reason.
> Long
> > > GC perhaps? But eventually the service should have recovered. Any
> > > indication of connection
> > > recovery later?
> > >
> > > Jerry
> > >
> > > On Sun, Dec 24, 2017 at 11:34 AM, Eddie Epstein <ea...@gmail.com>
> > > wrote:
> > >
> > > > Jerry,
> > > >
> > > > Testing a real analytic running 28 sync pipelines as a DUCC managed
> > > UIMA-AS
> > > > service, the DUCC pinger is reporting failures for an idle service 2
> > or 3
> > > > times a day.
> > > >
> > > > sm.log, an error surrounded by good pings, the ping normally done
> > every 2
> > > > minutes:
> > > >
> > > > Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
> > > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3]
> Prod[1]
> > > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > > > Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
> > > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3]
> Prod[1]
> > > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > > > Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
> > > > handleStatistics Missed_pings 1 endpoint
> > > > UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] +
> > > Info:
> > > > QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0]
> > inFlt[0]
> > > > DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta
> > to:
> > > > eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
> > > > Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
> > > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3]
> > > Prod[1]
> > > > minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > > >
> > > > The broker logfile, complaining about all three service listeners
> > around
> > > > the same time (the service runs on 192.168.7.101):
> > > >
> > > > 2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
> > > > - Transport Connection to: tcp://192.168.7.101:43895 failed:
> > > > org.apache.activemq.transport.InactivityIOException: Channel was
> > > inactive
> > > > for too (>30000) long: tcp://192.168.7.101:43895
> > > > 2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
> > > > - Transport Connection to: tcp://192.168.7.101:43898 failed:
> > > > org.apache.activemq.transport.InactivityIOException: Channel was
> > > inactive
> > > > for too (>30000) long: tcp://192.168.7.101:43898
> > > > 2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
> > > > - Transport Connection to: tcp://192.168.7.101:43899 failed:
> > > > org.apache.activemq.transport.InactivityIOException: Channel was
> > > inactive
> > > > for too (>30000) long: tcp://192.168.7.101:43899
> > > >
> > > > The service logfile also reporting problems for all three listeners:
> > > >
> > > > 07:55:12.457 - 91:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > > ainer.onException:
> > > > WARNING: Service: Processing2 Runtime Exception
> > > > 07:55:12.457 - 91:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > > ainer.onException:
> > > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> > java.io.EOFException
> > > > 07:55:12.457 - 91:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.
> > > > handleListenerSetupFailure:
> > > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > > Broker:
> > > > tcp://broker1:61616 Retrying Until Successful ...
> > > > 07:55:12.457 - 2336:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > > ainer.onException:
> > > > WARNING: Service: Processing2 Runtime Exception
> > > > 07:55:12.457 - 2336:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > > ainer.onException:
> > > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> > java.io.EOFException
> > > > 07:55:12.457 - 2335:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > > ainer.onException:
> > > > WARNING: Service: Processing2 Runtime Exception
> > > > 07:55:12.457 - 2335:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > > ainer.onException:
> > > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> > java.io.EOFException
> > > > 07:55:12.457 - 2336:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.
> > > > handleListenerSetupFailure:
> > > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > > Broker:
> > > > tcp://broker1:61616 Retrying Until Successful ...
> > > > 07:55:12.457 - 2335:
> > > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.
> > > > handleListenerSetupFailure:
> > > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > > Broker:
> > > > tcp://broker1:61616 Retrying Until Successful ...
> > > >
> > > >
> > > > sm.log is showing the same kind of ping failures for previous
> versions
> > of
> > > > UIMA-AS as well, not all at the same time as this event, but spread
> out
> > > > widely. The clocks on all three machines are synchronized so it could
> > be
> > > > meaningful that the pinger client registered the error before the
> > broker
> > > or
> > > > the service.
> > > >
> > > > Don't think this is a blocker, just wanted to check if there is a
> > problem
> > > > in UIMA-AS.
> > > >
> > > > Eddie
> > > >
> > > >
> > > > On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <
> cwiklik@apache.org>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
> > > > >
> > > > > This version contains the following fixes:
> > > > > - fixed documentation
> > > > >
> > > > > The list of changes in Jira:
> > > > > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> > > > > 202.10.2AS%20AND%20project%20%3D%20UIMA
> > > > >
> > > > > The source and binary zip/tars are staged to
> > > > > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
> > > > >
> > > > > The eclipse update subsite is here:
> > > > > http://dist.apache.org/repos/dist/dev/uima/eclipse-update-
> > > site/uima-as/
> > > > >
> > > > > The Maven artifacts are here:
> > > > > http://repository.apache.org/content/repositories/
> orgapacheuima-1177
> > > > >
> > > > > The SVN tags are here:
> > > > > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
> > > > >
> > > > > See http://uima.apache.org/testing-builds.html for suggestions on
> > how
> > > to
> > > > > test release candidates.
> > > > >
> > > > > Please vote on release:
> > > > >
> > > > > [ ] +1 OK to release
> > > > > [ ] 0 Don't care
> > > > > [ ] -1 Not OK to release, because ...
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Jaroslaw Cwiklik <cw...@apache.org>.
"Many other" = "all services " were kaput? If so, it could be the broker
failure (what version?) or something else. Sorry cant pinpoint this stuff
with accuracy since the broken connection can be symptom of many problems.
Is this system in the kaput state still?
Jerry
On Tue, Jan 2, 2018 at 11:16 AM, Eddie Epstein <ea...@gmail.com> wrote:
> There were no long GCs for this service. Note (from above) that many other
> services running older versions of uima-as were also showing the same
> behavior.
>
> Eddie
>
> On Tue, Jan 2, 2018 at 9:21 AM, Jaroslaw Cwiklik <cw...@apache.org>
> wrote:
>
> > Looks like the client lost connection to the broker for some reason. Long
> > GC perhaps? But eventually the service should have recovered. Any
> > indication of connection
> > recovery later?
> >
> > Jerry
> >
> > On Sun, Dec 24, 2017 at 11:34 AM, Eddie Epstein <ea...@gmail.com>
> > wrote:
> >
> > > Jerry,
> > >
> > > Testing a real analytic running 28 sync pipelines as a DUCC managed
> > UIMA-AS
> > > service, the DUCC pinger is reporting failures for an idle service 2
> or 3
> > > times a day.
> > >
> > > sm.log, an error surrounded by good pings, the ping normally done
> every 2
> > > minutes:
> > >
> > > Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > > Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > > Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Missed_pings 1 endpoint
> > > UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] +
> > Info:
> > > QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0]
> inFlt[0]
> > > DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta
> to:
> > > eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
> > > Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3]
> > Prod[1]
> > > minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > >
> > > The broker logfile, complaining about all three service listeners
> around
> > > the same time (the service runs on 192.168.7.101):
> > >
> > > 2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
> > > - Transport Connection to: tcp://192.168.7.101:43895 failed:
> > > org.apache.activemq.transport.InactivityIOException: Channel was
> > inactive
> > > for too (>30000) long: tcp://192.168.7.101:43895
> > > 2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
> > > - Transport Connection to: tcp://192.168.7.101:43898 failed:
> > > org.apache.activemq.transport.InactivityIOException: Channel was
> > inactive
> > > for too (>30000) long: tcp://192.168.7.101:43898
> > > 2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
> > > - Transport Connection to: tcp://192.168.7.101:43899 failed:
> > > org.apache.activemq.transport.InactivityIOException: Channel was
> > inactive
> > > for too (>30000) long: tcp://192.168.7.101:43899
> > >
> > > The service logfile also reporting problems for all three listeners:
> > >
> > > 07:55:12.457 - 91:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Service: Processing2 Runtime Exception
> > > 07:55:12.457 - 91:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> > > 07:55:12.457 - 91:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.
> > > handleListenerSetupFailure:
> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > Broker:
> > > tcp://broker1:61616 Retrying Until Successful ...
> > > 07:55:12.457 - 2336:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Service: Processing2 Runtime Exception
> > > 07:55:12.457 - 2336:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> > > 07:55:12.457 - 2335:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Service: Processing2 Runtime Exception
> > > 07:55:12.457 - 2335:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> > > 07:55:12.457 - 2336:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.
> > > handleListenerSetupFailure:
> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > Broker:
> > > tcp://broker1:61616 Retrying Until Successful ...
> > > 07:55:12.457 - 2335:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.
> > > handleListenerSetupFailure:
> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > Broker:
> > > tcp://broker1:61616 Retrying Until Successful ...
> > >
> > >
> > > sm.log is showing the same kind of ping failures for previous versions
> of
> > > UIMA-AS as well, not all at the same time as this event, but spread out
> > > widely. The clocks on all three machines are synchronized so it could
> be
> > > meaningful that the pinger client registered the error before the
> broker
> > or
> > > the service.
> > >
> > > Don't think this is a blocker, just wanted to check if there is a
> problem
> > > in UIMA-AS.
> > >
> > > Eddie
> > >
> > >
> > > On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
> > > >
> > > > This version contains the following fixes:
> > > > - fixed documentation
> > > >
> > > > The list of changes in Jira:
> > > > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> > > > 202.10.2AS%20AND%20project%20%3D%20UIMA
> > > >
> > > > The source and binary zip/tars are staged to
> > > > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
> > > >
> > > > The eclipse update subsite is here:
> > > > http://dist.apache.org/repos/dist/dev/uima/eclipse-update-
> > site/uima-as/
> > > >
> > > > The Maven artifacts are here:
> > > > http://repository.apache.org/content/repositories/orgapacheuima-1177
> > > >
> > > > The SVN tags are here:
> > > > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
> > > >
> > > > See http://uima.apache.org/testing-builds.html for suggestions on
> how
> > to
> > > > test release candidates.
> > > >
> > > > Please vote on release:
> > > >
> > > > [ ] +1 OK to release
> > > > [ ] 0 Don't care
> > > > [ ] -1 Not OK to release, because ...
> > > >
> > >
> >
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Jaroslaw Cwiklik <cw...@apache.org>.
Just to make this more clear, Eddie and I together determined that the AMQ
connection losses where due to
long GCs in the service. This conclusion was based on evidence found in the
service log and correlation with
jConsole stats.
Jerry
On Fri, Jan 5, 2018 at 10:56 AM, Jaroslaw Cwiklik <cw...@apache.org>
wrote:
> After reviewing the service log and looking at jConsole there are long GCs
> which cause connection loss and recovery.
> Looks like AE issue. Strange that these GCs happen if the service is idle.
> Someone should profile this AE to confirm
> where this and possibly fix the source of excessive garbage.
>
> Another possibility is to increase AMQ inactivityTimeout (default 30 secs)
> to prevent connection drops.
>
> Jerry
>
> On Tue, Jan 2, 2018 at 11:16 AM, Eddie Epstein <ea...@gmail.com>
> wrote:
>
>> There were no long GCs for this service. Note (from above) that many other
>> services running older versions of uima-as were also showing the same
>> behavior.
>>
>> Eddie
>>
>> On Tue, Jan 2, 2018 at 9:21 AM, Jaroslaw Cwiklik <cw...@apache.org>
>> wrote:
>>
>> > Looks like the client lost connection to the broker for some reason.
>> Long
>> > GC perhaps? But eventually the service should have recovered. Any
>> > indication of connection
>> > recovery later?
>> >
>> > Jerry
>> >
>> > On Sun, Dec 24, 2017 at 11:34 AM, Eddie Epstein <ea...@gmail.com>
>> > wrote:
>> >
>> > > Jerry,
>> > >
>> > > Testing a real analytic running 28 sync pipelines as a DUCC managed
>> > UIMA-AS
>> > > service, the DUCC pinger is reporting failures for an idle service 2
>> or 3
>> > > times a day.
>> > >
>> > > sm.log, an error surrounded by good pings, the ping normally done
>> every 2
>> > > minutes:
>> > >
>> > > Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
>> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
>> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
>> > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
>> > > MetaNode[192.168.7.101] MetaPid[71061:91]
>> > > Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
>> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
>> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
>> > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
>> > > MetaNode[192.168.7.101] MetaPid[71061:91]
>> > > Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
>> > > handleStatistics Missed_pings 1 endpoint
>> > > UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] +
>> > Info:
>> > > QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0]
>> inFlt[0]
>> > > DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta
>> to:
>> > > eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
>> > > Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
>> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
>> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3]
>> > Prod[1]
>> > > minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
>> > > MetaNode[192.168.7.101] MetaPid[71061:91]
>> > >
>> > > The broker logfile, complaining about all three service listeners
>> around
>> > > the same time (the service runs on 192.168.7.101):
>> > >
>> > > 2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
>> > > - Transport Connection to: tcp://192.168.7.101:43895 failed:
>> > > org.apache.activemq.transport.InactivityIOException: Channel was
>> > inactive
>> > > for too (>30000) long: tcp://192.168.7.101:43895
>> > > 2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
>> > > - Transport Connection to: tcp://192.168.7.101:43898 failed:
>> > > org.apache.activemq.transport.InactivityIOException: Channel was
>> > inactive
>> > > for too (>30000) long: tcp://192.168.7.101:43898
>> > > 2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
>> > > - Transport Connection to: tcp://192.168.7.101:43899 failed:
>> > > org.apache.activemq.transport.InactivityIOException: Channel was
>> > inactive
>> > > for too (>30000) long: tcp://192.168.7.101:43899
>> > >
>> > > The service logfile also reporting problems for all three listeners:
>> > >
>> > > 07:55:12.457 - 91:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > > ainer.onException:
>> > > WARNING: Service: Processing2 Runtime Exception
>> > > 07:55:12.457 - 91:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > > ainer.onException:
>> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
>> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> > > 07:55:12.457 - 91:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > ainer.
>> > > handleListenerSetupFailure:
>> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
>> > Broker:
>> > > tcp://broker1:61616 Retrying Until Successful ...
>> > > 07:55:12.457 - 2336:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > > ainer.onException:
>> > > WARNING: Service: Processing2 Runtime Exception
>> > > 07:55:12.457 - 2336:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > > ainer.onException:
>> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
>> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> > > 07:55:12.457 - 2335:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > > ainer.onException:
>> > > WARNING: Service: Processing2 Runtime Exception
>> > > 07:55:12.457 - 2335:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > > ainer.onException:
>> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
>> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
>> java.io.EOFException
>> > > 07:55:12.457 - 2336:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > ainer.
>> > > handleListenerSetupFailure:
>> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
>> > Broker:
>> > > tcp://broker1:61616 Retrying Until Successful ...
>> > > 07:55:12.457 - 2335:
>> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
>> > ainer.
>> > > handleListenerSetupFailure:
>> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
>> > Broker:
>> > > tcp://broker1:61616 Retrying Until Successful ...
>> > >
>> > >
>> > > sm.log is showing the same kind of ping failures for previous
>> versions of
>> > > UIMA-AS as well, not all at the same time as this event, but spread
>> out
>> > > widely. The clocks on all three machines are synchronized so it could
>> be
>> > > meaningful that the pinger client registered the error before the
>> broker
>> > or
>> > > the service.
>> > >
>> > > Don't think this is a blocker, just wanted to check if there is a
>> problem
>> > > in UIMA-AS.
>> > >
>> > > Eddie
>> > >
>> > >
>> > > On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cwiklik@apache.org
>> >
>> > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
>> > > >
>> > > > This version contains the following fixes:
>> > > > - fixed documentation
>> > > >
>> > > > The list of changes in Jira:
>> > > > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
>> > > > 202.10.2AS%20AND%20project%20%3D%20UIMA
>> > > >
>> > > > The source and binary zip/tars are staged to
>> > > > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
>> > > >
>> > > > The eclipse update subsite is here:
>> > > > http://dist.apache.org/repos/dist/dev/uima/eclipse-update-
>> > site/uima-as/
>> > > >
>> > > > The Maven artifacts are here:
>> > > > http://repository.apache.org/content/repositories/orgapacheu
>> ima-1177
>> > > >
>> > > > The SVN tags are here:
>> > > > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
>> > > >
>> > > > See http://uima.apache.org/testing-builds.html for suggestions on
>> how
>> > to
>> > > > test release candidates.
>> > > >
>> > > > Please vote on release:
>> > > >
>> > > > [ ] +1 OK to release
>> > > > [ ] 0 Don't care
>> > > > [ ] -1 Not OK to release, because ...
>> > > >
>> > >
>> >
>>
>
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Jaroslaw Cwiklik <cw...@apache.org>.
After reviewing the service log and looking at jConsole there are long GCs
which cause connection loss and recovery.
Looks like AE issue. Strange that these GCs happen if the service is idle.
Someone should profile this AE to confirm
where this and possibly fix the source of excessive garbage.
Another possibility is to increase AMQ inactivityTimeout (default 30 secs)
to prevent connection drops.
Jerry
On Tue, Jan 2, 2018 at 11:16 AM, Eddie Epstein <ea...@gmail.com> wrote:
> There were no long GCs for this service. Note (from above) that many other
> services running older versions of uima-as were also showing the same
> behavior.
>
> Eddie
>
> On Tue, Jan 2, 2018 at 9:21 AM, Jaroslaw Cwiklik <cw...@apache.org>
> wrote:
>
> > Looks like the client lost connection to the broker for some reason. Long
> > GC perhaps? But eventually the service should have recovered. Any
> > indication of connection
> > recovery later?
> >
> > Jerry
> >
> > On Sun, Dec 24, 2017 at 11:34 AM, Eddie Epstein <ea...@gmail.com>
> > wrote:
> >
> > > Jerry,
> > >
> > > Testing a real analytic running 28 sync pipelines as a DUCC managed
> > UIMA-AS
> > > service, the DUCC pinger is reporting failures for an idle service 2
> or 3
> > > times a day.
> > >
> > > sm.log, an error surrounded by good pings, the ping normally done
> every 2
> > > minutes:
> > >
> > > Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > > Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > > Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Missed_pings 1 endpoint
> > > UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] +
> > Info:
> > > QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0]
> inFlt[0]
> > > DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta
> to:
> > > eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
> > > Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
> > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3]
> > Prod[1]
> > > minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > > MetaNode[192.168.7.101] MetaPid[71061:91]
> > >
> > > The broker logfile, complaining about all three service listeners
> around
> > > the same time (the service runs on 192.168.7.101):
> > >
> > > 2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
> > > - Transport Connection to: tcp://192.168.7.101:43895 failed:
> > > org.apache.activemq.transport.InactivityIOException: Channel was
> > inactive
> > > for too (>30000) long: tcp://192.168.7.101:43895
> > > 2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
> > > - Transport Connection to: tcp://192.168.7.101:43898 failed:
> > > org.apache.activemq.transport.InactivityIOException: Channel was
> > inactive
> > > for too (>30000) long: tcp://192.168.7.101:43898
> > > 2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
> > > - Transport Connection to: tcp://192.168.7.101:43899 failed:
> > > org.apache.activemq.transport.InactivityIOException: Channel was
> > inactive
> > > for too (>30000) long: tcp://192.168.7.101:43899
> > >
> > > The service logfile also reporting problems for all three listeners:
> > >
> > > 07:55:12.457 - 91:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Service: Processing2 Runtime Exception
> > > 07:55:12.457 - 91:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> > > 07:55:12.457 - 91:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.
> > > handleListenerSetupFailure:
> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > Broker:
> > > tcp://broker1:61616 Retrying Until Successful ...
> > > 07:55:12.457 - 2336:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Service: Processing2 Runtime Exception
> > > 07:55:12.457 - 2336:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> > > 07:55:12.457 - 2335:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Service: Processing2 Runtime Exception
> > > 07:55:12.457 - 2335:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > > ainer.onException:
> > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > > tcp://broker1:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> > > 07:55:12.457 - 2336:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.
> > > handleListenerSetupFailure:
> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > Broker:
> > > tcp://broker1:61616 Retrying Until Successful ...
> > > 07:55:12.457 - 2335:
> > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.
> > > handleListenerSetupFailure:
> > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> > Broker:
> > > tcp://broker1:61616 Retrying Until Successful ...
> > >
> > >
> > > sm.log is showing the same kind of ping failures for previous versions
> of
> > > UIMA-AS as well, not all at the same time as this event, but spread out
> > > widely. The clocks on all three machines are synchronized so it could
> be
> > > meaningful that the pinger client registered the error before the
> broker
> > or
> > > the service.
> > >
> > > Don't think this is a blocker, just wanted to check if there is a
> problem
> > > in UIMA-AS.
> > >
> > > Eddie
> > >
> > >
> > > On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
> > > >
> > > > This version contains the following fixes:
> > > > - fixed documentation
> > > >
> > > > The list of changes in Jira:
> > > > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> > > > 202.10.2AS%20AND%20project%20%3D%20UIMA
> > > >
> > > > The source and binary zip/tars are staged to
> > > > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
> > > >
> > > > The eclipse update subsite is here:
> > > > http://dist.apache.org/repos/dist/dev/uima/eclipse-update-
> > site/uima-as/
> > > >
> > > > The Maven artifacts are here:
> > > > http://repository.apache.org/content/repositories/orgapacheuima-1177
> > > >
> > > > The SVN tags are here:
> > > > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
> > > >
> > > > See http://uima.apache.org/testing-builds.html for suggestions on
> how
> > to
> > > > test release candidates.
> > > >
> > > > Please vote on release:
> > > >
> > > > [ ] +1 OK to release
> > > > [ ] 0 Don't care
> > > > [ ] -1 Not OK to release, because ...
> > > >
> > >
> >
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Eddie Epstein <ea...@gmail.com>.
There were no long GCs for this service. Note (from above) that many other
services running older versions of uima-as were also showing the same
behavior.
Eddie
On Tue, Jan 2, 2018 at 9:21 AM, Jaroslaw Cwiklik <cw...@apache.org> wrote:
> Looks like the client lost connection to the broker for some reason. Long
> GC perhaps? But eventually the service should have recovered. Any
> indication of connection
> recovery later?
>
> Jerry
>
> On Sun, Dec 24, 2017 at 11:34 AM, Eddie Epstein <ea...@gmail.com>
> wrote:
>
> > Jerry,
> >
> > Testing a real analytic running 28 sync pipelines as a DUCC managed
> UIMA-AS
> > service, the DUCC pinger is reporting failures for an idle service 2 or 3
> > times a day.
> >
> > sm.log, an error surrounded by good pings, the ping normally done every 2
> > minutes:
> >
> > Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
> > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > MetaNode[192.168.7.101] MetaPid[71061:91]
> > Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
> > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > MetaNode[192.168.7.101] MetaPid[71061:91]
> > Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
> > handleStatistics Missed_pings 1 endpoint
> > UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] +
> Info:
> > QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0] inFlt[0]
> > DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta to:
> > eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
> > Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
> > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3]
> Prod[1]
> > minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> > MetaNode[192.168.7.101] MetaPid[71061:91]
> >
> > The broker logfile, complaining about all three service listeners around
> > the same time (the service runs on 192.168.7.101):
> >
> > 2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
> > - Transport Connection to: tcp://192.168.7.101:43895 failed:
> > org.apache.activemq.transport.InactivityIOException: Channel was
> inactive
> > for too (>30000) long: tcp://192.168.7.101:43895
> > 2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
> > - Transport Connection to: tcp://192.168.7.101:43898 failed:
> > org.apache.activemq.transport.InactivityIOException: Channel was
> inactive
> > for too (>30000) long: tcp://192.168.7.101:43898
> > 2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
> > - Transport Connection to: tcp://192.168.7.101:43899 failed:
> > org.apache.activemq.transport.InactivityIOException: Channel was
> inactive
> > for too (>30000) long: tcp://192.168.7.101:43899
> >
> > The service logfile also reporting problems for all three listeners:
> >
> > 07:55:12.457 - 91:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.onException:
> > WARNING: Service: Processing2 Runtime Exception
> > 07:55:12.457 - 91:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.onException:
> > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
> > 07:55:12.457 - 91:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> > handleListenerSetupFailure:
> > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> Broker:
> > tcp://broker1:61616 Retrying Until Successful ...
> > 07:55:12.457 - 2336:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.onException:
> > WARNING: Service: Processing2 Runtime Exception
> > 07:55:12.457 - 2336:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.onException:
> > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
> > 07:55:12.457 - 2335:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.onException:
> > WARNING: Service: Processing2 Runtime Exception
> > 07:55:12.457 - 2335:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> > ainer.onException:
> > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> > tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
> > 07:55:12.457 - 2336:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> > handleListenerSetupFailure:
> > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> Broker:
> > tcp://broker1:61616 Retrying Until Successful ...
> > 07:55:12.457 - 2335:
> > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.
> > handleListenerSetupFailure:
> > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To
> Broker:
> > tcp://broker1:61616 Retrying Until Successful ...
> >
> >
> > sm.log is showing the same kind of ping failures for previous versions of
> > UIMA-AS as well, not all at the same time as this event, but spread out
> > widely. The clocks on all three machines are synchronized so it could be
> > meaningful that the pinger client registered the error before the broker
> or
> > the service.
> >
> > Don't think this is a blocker, just wanted to check if there is a problem
> > in UIMA-AS.
> >
> > Eddie
> >
> >
> > On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
> > wrote:
> >
> > > Hi,
> > >
> > > the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
> > >
> > > This version contains the following fixes:
> > > - fixed documentation
> > >
> > > The list of changes in Jira:
> > > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> > > 202.10.2AS%20AND%20project%20%3D%20UIMA
> > >
> > > The source and binary zip/tars are staged to
> > > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
> > >
> > > The eclipse update subsite is here:
> > > http://dist.apache.org/repos/dist/dev/uima/eclipse-update-
> site/uima-as/
> > >
> > > The Maven artifacts are here:
> > > http://repository.apache.org/content/repositories/orgapacheuima-1177
> > >
> > > The SVN tags are here:
> > > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
> > >
> > > See http://uima.apache.org/testing-builds.html for suggestions on how
> to
> > > test release candidates.
> > >
> > > Please vote on release:
> > >
> > > [ ] +1 OK to release
> > > [ ] 0 Don't care
> > > [ ] -1 Not OK to release, because ...
> > >
> >
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Jaroslaw Cwiklik <cw...@apache.org>.
Looks like the client lost connection to the broker for some reason. Long
GC perhaps? But eventually the service should have recovered. Any
indication of connection
recovery later?
Jerry
On Sun, Dec 24, 2017 at 11:34 AM, Eddie Epstein <ea...@gmail.com> wrote:
> Jerry,
>
> Testing a real analytic running 28 sync pipelines as a DUCC managed UIMA-AS
> service, the DUCC pinger is reporting failures for an idle service 2 or 3
> times a day.
>
> sm.log, an error surrounded by good pings, the ping normally done every 2
> minutes:
>
> Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
> handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> MetaNode[192.168.7.101] MetaPid[71061:91]
> Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
> handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
> minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> MetaNode[192.168.7.101] MetaPid[71061:91]
> Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
> handleStatistics Missed_pings 1 endpoint
> UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] + Info:
> QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0] inFlt[0]
> DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta to:
> eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
> Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
> handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
> Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3] Prod[1]
> minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
> MetaNode[192.168.7.101] MetaPid[71061:91]
>
> The broker logfile, complaining about all three service listeners around
> the same time (the service runs on 192.168.7.101):
>
> 2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
> - Transport Connection to: tcp://192.168.7.101:43895 failed:
> org.apache.activemq.transport.InactivityIOException: Channel was inactive
> for too (>30000) long: tcp://192.168.7.101:43895
> 2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
> - Transport Connection to: tcp://192.168.7.101:43898 failed:
> org.apache.activemq.transport.InactivityIOException: Channel was inactive
> for too (>30000) long: tcp://192.168.7.101:43898
> 2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
> - Transport Connection to: tcp://192.168.7.101:43899 failed:
> org.apache.activemq.transport.InactivityIOException: Channel was inactive
> for too (>30000) long: tcp://192.168.7.101:43899
>
> The service logfile also reporting problems for all three listeners:
>
> 07:55:12.457 - 91:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Processing2 Runtime Exception
> 07:55:12.457 - 91:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
> 07:55:12.457 - 91:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
> tcp://broker1:61616 Retrying Until Successful ...
> 07:55:12.457 - 2336:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Processing2 Runtime Exception
> 07:55:12.457 - 2336:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
> 07:55:12.457 - 2335:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Processing2 Runtime Exception
> 07:55:12.457 - 2335:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
> tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
> 07:55:12.457 - 2336:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
> tcp://broker1:61616 Retrying Until Successful ...
> 07:55:12.457 - 2335:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
> tcp://broker1:61616 Retrying Until Successful ...
>
>
> sm.log is showing the same kind of ping failures for previous versions of
> UIMA-AS as well, not all at the same time as this event, but spread out
> widely. The clocks on all three machines are synchronized so it could be
> meaningful that the pinger client registered the error before the broker or
> the service.
>
> Don't think this is a blocker, just wanted to check if there is a problem
> in UIMA-AS.
>
> Eddie
>
>
> On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
> wrote:
>
> > Hi,
> >
> > the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
> >
> > This version contains the following fixes:
> > - fixed documentation
> >
> > The list of changes in Jira:
> > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> > 202.10.2AS%20AND%20project%20%3D%20UIMA
> >
> > The source and binary zip/tars are staged to
> > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
> >
> > The eclipse update subsite is here:
> > http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
> >
> > The Maven artifacts are here:
> > http://repository.apache.org/content/repositories/orgapacheuima-1177
> >
> > The SVN tags are here:
> > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
> >
> > See http://uima.apache.org/testing-builds.html for suggestions on how to
> > test release candidates.
> >
> > Please vote on release:
> >
> > [ ] +1 OK to release
> > [ ] 0 Don't care
> > [ ] -1 Not OK to release, because ...
> >
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Eddie Epstein <ea...@gmail.com>.
Jerry,
Testing a real analytic running 28 sync pipelines as a DUCC managed UIMA-AS
service, the DUCC pinger is reporting failures for an idle service 2 or 3
times a day.
sm.log, an error surrounded by good pings, the ping normally done every 2
minutes:
Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
MetaNode[192.168.7.101] MetaPid[71061:91]
Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
MetaNode[192.168.7.101] MetaPid[71061:91]
Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
handleStatistics Missed_pings 1 endpoint
UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] + Info:
QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0] inFlt[0]
DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta to:
eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3] Prod[1]
minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
MetaNode[192.168.7.101] MetaPid[71061:91]
The broker logfile, complaining about all three service listeners around
the same time (the service runs on 192.168.7.101):
2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
- Transport Connection to: tcp://192.168.7.101:43895 failed:
org.apache.activemq.transport.InactivityIOException: Channel was inactive
for too (>30000) long: tcp://192.168.7.101:43895
2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
- Transport Connection to: tcp://192.168.7.101:43898 failed:
org.apache.activemq.transport.InactivityIOException: Channel was inactive
for too (>30000) long: tcp://192.168.7.101:43898
2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
- Transport Connection to: tcp://192.168.7.101:43899 failed:
org.apache.activemq.transport.InactivityIOException: Channel was inactive
for too (>30000) long: tcp://192.168.7.101:43899
The service logfile also reporting problems for all three listeners:
07:55:12.457 - 91:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Processing2 Runtime Exception
07:55:12.457 - 91:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
07:55:12.457 - 91:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
tcp://broker1:61616 Retrying Until Successful ...
07:55:12.457 - 2336:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Processing2 Runtime Exception
07:55:12.457 - 2336:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
07:55:12.457 - 2335:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Processing2 Runtime Exception
07:55:12.457 - 2335:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
07:55:12.457 - 2336:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
tcp://broker1:61616 Retrying Until Successful ...
07:55:12.457 - 2335:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
tcp://broker1:61616 Retrying Until Successful ...
sm.log is showing the same kind of ping failures for previous versions of
UIMA-AS as well, not all at the same time as this event, but spread out
widely. The clocks on all three machines are synchronized so it could be
meaningful that the pinger client registered the error before the broker or
the service.
Don't think this is a blocker, just wanted to check if there is a problem
in UIMA-AS.
Eddie
On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
wrote:
> Hi,
>
> the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
>
> This version contains the following fixes:
> - fixed documentation
>
> The list of changes in Jira:
> http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> 202.10.2AS%20AND%20project%20%3D%20UIMA
>
> The source and binary zip/tars are staged to
> http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
>
> The eclipse update subsite is here:
> http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
>
> The Maven artifacts are here:
> http://repository.apache.org/content/repositories/orgapacheuima-1177
>
> The SVN tags are here:
> http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
>
> See http://uima.apache.org/testing-builds.html for suggestions on how to
> test release candidates.
>
> Please vote on release:
>
> [ ] +1 OK to release
> [ ] 0 Don't care
> [ ] -1 Not OK to release, because ...
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Marshall Schor <ms...@schor.com>.
Hi Jerry,
How about checking your release script for uima-as into svn as part of the
uima-as project?
Benefits- it won't get lost, and it's available to others who might someday need
to release uima-as.
Cheers. -Marshall
On 1/7/2018 6:27 PM, Jaroslaw Cwiklik wrote:
> Damn, I now remember this exact issue biting me before. I'd modified my old
> release script but
> unfortunately lost it when my disk drive died. Will update the new one to
> make sure this is always
> tested with every release.
>
> Jerry
>
> On Sat, Jan 6, 2018 at 6:55 PM, Eddie Epstein <ea...@gmail.com> wrote:
>
>> - downloaded binary tarball and checked md5sum
>> - unpacked, set UIMA_HOME, updated PATH and ran adjustExamplePaths.sh
>> - startBroker.sh worked normally
>> - ran basic test (MeetingDetectorTaeQueue) from README
>> - confirmed -log and -perf options on the basic test are working
>> (although error message when required perf-log-file not specified should be
>> improved)
>>
>> - downloaded source zip and checked md5sum
>> - built as per embedded documentation but this failed. Adding -U to "mvn
>> clean install" fixed it.
>>
>> - imported uimaj-examples project and defined UIMA_HOME classpath variable
>> - tried to run "UIMA Deploy AS Service" but this failed because,
>> - the classpath includes the previous version of AMQ and spring jars,
>> and even after fixing that,
>> - "Project" is unset in the run configs.
>>
>> [ ] -1 Not OK to release, because of problems with the uimaj-examples
>> project.
>>
>> Eddie
>>
>>
>> On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
>>>
>>> This version contains the following fixes:
>>> - fixed documentation
>>>
>>> The list of changes in Jira:
>>> http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
>>> 202.10.2AS%20AND%20project%20%3D%20UIMA
>>>
>>> The source and binary zip/tars are staged to
>>> http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
>>>
>>> The eclipse update subsite is here:
>>> http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
>>>
>>> The Maven artifacts are here:
>>> http://repository.apache.org/content/repositories/orgapacheuima-1177
>>>
>>> The SVN tags are here:
>>> http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
>>>
>>> See http://uima.apache.org/testing-builds.html for suggestions on how to
>>> test release candidates.
>>>
>>> Please vote on release:
>>>
>>> [ ] +1 OK to release
>>> [ ] 0 Don't care
>>> [ ] -1 Not OK to release, because ...
>>>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Jaroslaw Cwiklik <cw...@apache.org>.
Damn, I now remember this exact issue biting me before. I'd modified my old
release script but
unfortunately lost it when my disk drive died. Will update the new one to
make sure this is always
tested with every release.
Jerry
On Sat, Jan 6, 2018 at 6:55 PM, Eddie Epstein <ea...@gmail.com> wrote:
> - downloaded binary tarball and checked md5sum
> - unpacked, set UIMA_HOME, updated PATH and ran adjustExamplePaths.sh
> - startBroker.sh worked normally
> - ran basic test (MeetingDetectorTaeQueue) from README
> - confirmed -log and -perf options on the basic test are working
> (although error message when required perf-log-file not specified should be
> improved)
>
> - downloaded source zip and checked md5sum
> - built as per embedded documentation but this failed. Adding -U to "mvn
> clean install" fixed it.
>
> - imported uimaj-examples project and defined UIMA_HOME classpath variable
> - tried to run "UIMA Deploy AS Service" but this failed because,
> - the classpath includes the previous version of AMQ and spring jars,
> and even after fixing that,
> - "Project" is unset in the run configs.
>
> [ ] -1 Not OK to release, because of problems with the uimaj-examples
> project.
>
> Eddie
>
>
> On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
> wrote:
>
> > Hi,
> >
> > the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
> >
> > This version contains the following fixes:
> > - fixed documentation
> >
> > The list of changes in Jira:
> > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> > 202.10.2AS%20AND%20project%20%3D%20UIMA
> >
> > The source and binary zip/tars are staged to
> > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
> >
> > The eclipse update subsite is here:
> > http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
> >
> > The Maven artifacts are here:
> > http://repository.apache.org/content/repositories/orgapacheuima-1177
> >
> > The SVN tags are here:
> > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
> >
> > See http://uima.apache.org/testing-builds.html for suggestions on how to
> > test release candidates.
> >
> > Please vote on release:
> >
> > [ ] +1 OK to release
> > [ ] 0 Don't care
> > [ ] -1 Not OK to release, because ...
> >
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Eddie Epstein <ea...@gmail.com>.
- downloaded binary tarball and checked md5sum
- unpacked, set UIMA_HOME, updated PATH and ran adjustExamplePaths.sh
- startBroker.sh worked normally
- ran basic test (MeetingDetectorTaeQueue) from README
- confirmed -log and -perf options on the basic test are working
(although error message when required perf-log-file not specified should be
improved)
- downloaded source zip and checked md5sum
- built as per embedded documentation but this failed. Adding -U to "mvn
clean install" fixed it.
- imported uimaj-examples project and defined UIMA_HOME classpath variable
- tried to run "UIMA Deploy AS Service" but this failed because,
- the classpath includes the previous version of AMQ and spring jars,
and even after fixing that,
- "Project" is unset in the run configs.
[ ] -1 Not OK to release, because of problems with the uimaj-examples
project.
Eddie
On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
wrote:
> Hi,
>
> the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
>
> This version contains the following fixes:
> - fixed documentation
>
> The list of changes in Jira:
> http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> 202.10.2AS%20AND%20project%20%3D%20UIMA
>
> The source and binary zip/tars are staged to
> http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
>
> The eclipse update subsite is here:
> http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
>
> The Maven artifacts are here:
> http://repository.apache.org/content/repositories/orgapacheuima-1177
>
> The SVN tags are here:
> http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
>
> See http://uima.apache.org/testing-builds.html for suggestions on how to
> test release candidates.
>
> Please vote on release:
>
> [ ] +1 OK to release
> [ ] 0 Don't care
> [ ] -1 Not OK to release, because ...
>
Re: [VOTE] UIMA-AS 2.10.2 RC5
Posted by Jaroslaw Cwiklik <cw...@apache.org>.
- Built from source - OK
- Spot checked signatures - OK
- issues fixed - OK
- Checked README, RELEASE_NOTES,LICENSE and NOTICE - OK
- Documentation (ducc book, javadoc api) - OK
- Installed eclipse plugins in eclipse Mars 2 - OK
[ ] +1 OK to release
Jerry
On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cw...@apache.org>
wrote:
> Hi,
>
> the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
>
> This version contains the following fixes:
> - fixed documentation
>
> The list of changes in Jira:
> http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> 202.10.2AS%20AND%20project%20%3D%20UIMA
>
> The source and binary zip/tars are staged to
> http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
>
> The eclipse update subsite is here:
> http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
>
> The Maven artifacts are here:
> http://repository.apache.org/content/repositories/orgapacheuima-1177
>
> The SVN tags are here:
> http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
>
> See http://uima.apache.org/testing-builds.html for suggestions on how to
> test release candidates.
>
> Please vote on release:
>
> [ ] +1 OK to release
> [ ] 0 Don't care
> [ ] -1 Not OK to release, because ...
>