You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by Ihsan Ecemis <mi...@gmail.com> on 2019/10/01 01:53:15 UTC
Re: Problem with MessageConsumers under TomEE 8.0.0
Thanks for digging into this Jon (and pointing my attention toward that other paragraph in the Javadoc)
And I highly appreciated the try-with-resources workaround.
> On Sep 30, 2019, at 17:11, Jonathan Gallimore <jo...@gmail.com> wrote:
>
>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
> states the following: There is no need to close the sessions, producers,
> and consumers of a closed connection.
>
>> Here is the link:
> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-- <
> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
>
> Good question. The changes in this area between 8.0.0-M3 and 8.0.0, are,
> you'll be unsurprised to find out, around transaction handling. We need to
> dig in a bit further.
>
> The Javadoc also states this: "Closing a connection causes any of its
> sessions' transactions in progress to be rolled back. In the case where a
> session's work is coordinated by an external transaction manager, a
> session's commit and rollback methods are not used and the result of a
> closed session's work is determined later by the transaction manager.
> Closing a connection does NOT force an acknowledgment of
> client-acknowledged sessions."
>
> A lot of this describes standalone usage, as opposed to usage in an
> application server. The Connection object should actually be a proxy, and
> calling close() shouldn't actually do anything. The sendMessage() method on
> the CustomJmsService bean is transactional, and the connection should be
> returned back to the pool once the method completes and the transaction is
> committed.
>
> Thank you for the sample code, we should be able to debug through and see
> what's going on. I should be able to do this over the next day - I'll post
> my steps here in case you want to dig in and have a go yourself.
>
> Jon
>
> On Mon, Sep 30, 2019 at 6:32 PM Ihsan Ecemis <mi...@gmail.com> wrote:
>
>>
>> Thank you very much for your suggestion Jon, adding
>> connection.createSession(), session.createProducer(), and
>> session.createConsumer() in the try-with-resources block make things work
>> again!
>>
>>
>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
>> states the following: There is no need to close the sessions, producers,
>> and consumers of a closed connection.
>>
>> Here is the link:
>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-- <
>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
>>
>>
>> And our original code, that did not close/auto-close Session,
>> MessageProducer, and MessageConsumer objects worked fine since at least
>> TomEE 7.0.2 (our git history shows that we had such a piece of code
>> implemented in March 2017)
>>
>>
>>
>>> On Sep 30, 2019, at 09:14, Jonathan Gallimore <
>> jonathan.gallimore@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I'm wondering if this is because your send method isn't closing the
>>> producer and the session? I did try your code and I saw the issue. I
>> turned
>>> it into an example, and added an Arquillian test:
>>> https://github.com/apache/tomee/pull/578
>>>
>>> Note that I added the session, producer and consumer into the
>>> try-with-resources block so they are auto-closed at the end of the
>> method.
>>>
>>> Can you take a look and let me know what you think?
>>>
>>> Jon
>>>
>>> On Fri, Sep 27, 2019 at 6:05 PM Ihsan Ecemis <mi...@gmail.com> wrote:
>>>
>>>>
>>>> Hello Everyone,
>>>>
>>>> We recently upgraded our staging environment from TomEE 8.0.0-M3 to
>> 8.0.0
>>>> and things started to break.
>>>>
>>>> After some troubleshooting, we realized that we cannot dequeue JMS
>>>> messages from our ActiveMQ server using MessageConsumers.
>>>>
>>>>
>>>> We dequeue messages in 2 different ways: We use MessageDriven beans for
>>>> most of our queues. But with some others, we periodically poll by
>> creating
>>>> a MessageConsumer (please see the code below for that second case)
>>>>
>>>> MessageDriven beans work without any problems but we cannot receive any
>>>> messages via MessageConsumers. That same backend code is working fine
>> with
>>>> 8.0.0-M3.
>>>>
>>>>
>>>> We tested this with different versions of ActiveMQ server (5.15.6,
>> 5.15.9,
>>>> 5.15.10) but TomEE 8.0.0 did not work with any of them.
>>>>
>>>>
>>>> Below is redacted code snippets showing where we have the problem.
>>>>
>>>> Any help will be greatly appreciated. Please let me know if you have
>> any
>>>> questions about our setup.
>>>>
>>>> Thanks,
>>>>
>>>> Ihsan.
>>>>
>>>>
>>>>
>>>> @Stateless
>>>> public class LogService {
>>>>
>>>> @EJB
>>>> private CustomJmsService customJmsService;
>>>>
>>>> @javax.ejb.Schedule(second = "*/30", minute = "*", hour = "*")
>>>> public void pollLogQueue() throws Exception {
>>>> final TextMessage logMessage =
>>>> customJmsService.receiveLogMessage(1000);
>>>> if (logMessage != null) {
>>>> persistLogMessages(logMessages);
>>>> }
>>>> }
>>>> }
>>>>
>>>> @Stateless
>>>> public class CustomJmsService {
>>>>
>>>> @Resource(name = "logQueue")
>>>> private Queue logQueue;
>>>>
>>>> public void sendLogMessage(final LogMessage message) {
>>>> sendMessage(logQueue, message);
>>>> }
>>>>
>>>> private void sendMessage(final Queue queue, final CustomJmsMessage
>>>> message) {
>>>> try (final Connection connection =
>>>> connectionFactory.createConnection()) {
>>>> connection.start();
>>>>
>>>> final Session session = connection.createSession(true,
>>>> Session.AUTO_ACKNOWLEDGE);
>>>> final MessageProducer producer =
>> session.createProducer(queue);
>>>> final String serializedMessage =
>>>> CustomJsonProvider.toJson(message);
>>>> final Message jmsMessage =
>>>> session.createTextMessage(serializedMessage);
>>>>
>>>> // This enqueues messages successfully with both 8.0.0-M3 and
>>>> 8.0.0
>>>> producer.send(jmsMessage);
>>>> } catch (final Exception e) {
>>>> throw new RuntimeException("Caught exception from JMS when
>>>> sending a message", e);
>>>> }
>>>> }
>>>>
>>>> public TextMessage receiveLogMessage(final long
>> receiveTimeoutMillis) {
>>>> return receiveMessage(logQueue, receiveTimeoutMillis);
>>>> }
>>>>
>>>> private TextMessage receiveMessage(final Queue queue, final long
>>>> receiveTimeoutMillis) {
>>>> try (final Connection connection =
>>>> connectionFactory.createConnection()) {
>>>> connection.start();
>>>>
>>>> final Session session = connection.createSession(true,
>>>> Session.AUTO_ACKNOWLEDGE);
>>>> final MessageConsumer messageConsumer =
>>>> session.createConsumer(queue);
>>>> final Message jmsMessage =
>>>> messageConsumer.receive(receiveTimeoutMillis);
>>>>
>>>> // PROBLEM: jmsMessage is always null with 8.0.0. This was
>>>> working with 8.0.0-M3
>>>> if (jmsMessage == null) {
>>>> return null;
>>>> }
>>>>
>>>> return (TextMessage) jmsMessage;
>>>> } catch (final Exception e) {
>>>> throw new RuntimeException("Caught exception from JMS when
>>>> receiving a message", e);
>>>> }
>>>> }
>>>> }
>>>>
>>>>
>>>>
>>
>>
Re: Problem with MessageConsumers under TomEE 8.0.0
Posted by "Jonathan S. Fisher" <ex...@gmail.com>.
Indeed, that is very strange. Setting transactionSupport = none essentially
reverts the server to its previous behavior.
On Thu, Oct 10, 2019 at 12:32 PM Jonathan Gallimore <
jonathan.gallimore@gmail.com> wrote:
> That's interesting. We'll see if we can create some test cases for these
> different scenarios.
>
> Jon
>
> On Thu, Oct 10, 2019 at 4:55 PM Ihsan Ecemis <mi...@gmail.com> wrote:
>
> >
> > Sorry I just had time to make these experiments.
> >
> > Having the following (setting transacted = false) did not fix the problem
> > if we do not use try-with-resources:
> >
> > final Session session = connection.createSession(false,
> > Session.AUTO_ACKNOWLEDGE);
> >
> >
> > On the other hand, when we added transactionSupport=none to
> resources.xml,
> > we could dequeue only 1 message, which is weird (at each server restart,
> > we dequeue 1 message even though there are more message in the queue.
> > Reverting the code back to try-with-resources makes the server dequeue
> all
> > the messages)
> >
> >
> > Hope that would help you guys find out where the problem is.
> >
> >
> >
> > > On Oct 1, 2019, at 16:42, Jonathan S. Fisher <ex...@gmail.com>
> wrote:
> > >
> > > Just curious, try this:
> > >
> > > final Session session = connection.createSession(false,
> > > Session.AUTO_ACKNOWLEDGE);
> > >
> > > and/or try this:
> > >
> > > <Resource id="MyJmsConnectionFactory"
> type="javax.jms.ConnectionFactory">
> > > ResourceAdapter = MyJmsResourceAdapter
> > > transactionSupport=none
> > > </Resource>
> > >
> > >
> > >
> > >
> > > On Mon, Sep 30, 2019 at 8:53 PM Ihsan Ecemis <mi...@gmail.com>
> wrote:
> > >
> > >>
> > >> Thanks for digging into this Jon (and pointing my attention toward
> that
> > >> other paragraph in the Javadoc)
> > >>
> > >> And I highly appreciated the try-with-resources workaround.
> > >>
> > >>
> > >>> On Sep 30, 2019, at 17:11, Jonathan Gallimore <
> > >> jonathan.gallimore@gmail.com> wrote:
> > >>>
> > >>>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s
> Javadoc
> > >>> states the following: There is no need to close the sessions,
> > producers,
> > >>> and consumers of a closed connection.
> > >>>
> > >>>> Here is the link:
> > >>>
> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
> > <
> > >>>
> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
> > >
> > >>>
> > >>> Good question. The changes in this area between 8.0.0-M3 and 8.0.0,
> > are,
> > >>> you'll be unsurprised to find out, around transaction handling. We
> need
> > >> to
> > >>> dig in a bit further.
> > >>>
> > >>> The Javadoc also states this: "Closing a connection causes any of its
> > >>> sessions' transactions in progress to be rolled back. In the case
> > where a
> > >>> session's work is coordinated by an external transaction manager, a
> > >>> session's commit and rollback methods are not used and the result of
> a
> > >>> closed session's work is determined later by the transaction manager.
> > >>> Closing a connection does NOT force an acknowledgment of
> > >>> client-acknowledged sessions."
> > >>>
> > >>> A lot of this describes standalone usage, as opposed to usage in an
> > >>> application server. The Connection object should actually be a proxy,
> > and
> > >>> calling close() shouldn't actually do anything. The sendMessage()
> > method
> > >> on
> > >>> the CustomJmsService bean is transactional, and the connection should
> > be
> > >>> returned back to the pool once the method completes and the
> transaction
> > >> is
> > >>> committed.
> > >>>
> > >>> Thank you for the sample code, we should be able to debug through and
> > see
> > >>> what's going on. I should be able to do this over the next day - I'll
> > >> post
> > >>> my steps here in case you want to dig in and have a go yourself.
> > >>>
> > >>> Jon
> > >>>
> > >>> On Mon, Sep 30, 2019 at 6:32 PM Ihsan Ecemis <mi...@gmail.com>
> > wrote:
> > >>>
> > >>>>
> > >>>> Thank you very much for your suggestion Jon, adding
> > >>>> connection.createSession(), session.createProducer(), and
> > >>>> session.createConsumer() in the try-with-resources block make
> things
> > >> work
> > >>>> again!
> > >>>>
> > >>>>
> > >>>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s
> Javadoc
> > >>>> states the following: There is no need to close the sessions,
> > >> producers,
> > >>>> and consumers of a closed connection.
> > >>>>
> > >>>> Here is the link:
> > >>>>
> > https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
> > >> <
> > >>>>
> > https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
> > >>>>
> > >>>>
> > >>>> And our original code, that did not close/auto-close Session,
> > >>>> MessageProducer, and MessageConsumer objects worked fine since at
> > least
> > >>>> TomEE 7.0.2 (our git history shows that we had such a piece of code
> > >>>> implemented in March 2017)
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On Sep 30, 2019, at 09:14, Jonathan Gallimore <
> > >>>> jonathan.gallimore@gmail.com> wrote:
> > >>>>>
> > >>>>> Hi,
> > >>>>>
> > >>>>> I'm wondering if this is because your send method isn't closing the
> > >>>>> producer and the session? I did try your code and I saw the issue.
> I
> > >>>> turned
> > >>>>> it into an example, and added an Arquillian test:
> > >>>>> https://github.com/apache/tomee/pull/578
> > >>>>>
> > >>>>> Note that I added the session, producer and consumer into the
> > >>>>> try-with-resources block so they are auto-closed at the end of the
> > >>>> method.
> > >>>>>
> > >>>>> Can you take a look and let me know what you think?
> > >>>>>
> > >>>>> Jon
> > >>>>>
> > >>>>> On Fri, Sep 27, 2019 at 6:05 PM Ihsan Ecemis <mi...@gmail.com>
> > >> wrote:
> > >>>>>
> > >>>>>>
> > >>>>>> Hello Everyone,
> > >>>>>>
> > >>>>>> We recently upgraded our staging environment from TomEE 8.0.0-M3
> to
> > >>>> 8.0.0
> > >>>>>> and things started to break.
> > >>>>>>
> > >>>>>> After some troubleshooting, we realized that we cannot dequeue JMS
> > >>>>>> messages from our ActiveMQ server using MessageConsumers.
> > >>>>>>
> > >>>>>>
> > >>>>>> We dequeue messages in 2 different ways: We use MessageDriven
> beans
> > >> for
> > >>>>>> most of our queues. But with some others, we periodically poll by
> > >>>> creating
> > >>>>>> a MessageConsumer (please see the code below for that second case)
> > >>>>>>
> > >>>>>> MessageDriven beans work without any problems but we cannot
> receive
> > >> any
> > >>>>>> messages via MessageConsumers. That same backend code is working
> > fine
> > >>>> with
> > >>>>>> 8.0.0-M3.
> > >>>>>>
> > >>>>>>
> > >>>>>> We tested this with different versions of ActiveMQ server (5.15.6,
> > >>>> 5.15.9,
> > >>>>>> 5.15.10) but TomEE 8.0.0 did not work with any of them.
> > >>>>>>
> > >>>>>>
> > >>>>>> Below is redacted code snippets showing where we have the problem.
> > >>>>>>
> > >>>>>> Any help will be greatly appreciated. Please let me know if you
> > have
> > >>>> any
> > >>>>>> questions about our setup.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Ihsan.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> @Stateless
> > >>>>>> public class LogService {
> > >>>>>>
> > >>>>>> @EJB
> > >>>>>> private CustomJmsService customJmsService;
> > >>>>>>
> > >>>>>> @javax.ejb.Schedule(second = "*/30", minute = "*", hour = "*")
> > >>>>>> public void pollLogQueue() throws Exception {
> > >>>>>> final TextMessage logMessage =
> > >>>>>> customJmsService.receiveLogMessage(1000);
> > >>>>>> if (logMessage != null) {
> > >>>>>> persistLogMessages(logMessages);
> > >>>>>> }
> > >>>>>> }
> > >>>>>> }
> > >>>>>>
> > >>>>>> @Stateless
> > >>>>>> public class CustomJmsService {
> > >>>>>>
> > >>>>>> @Resource(name = "logQueue")
> > >>>>>> private Queue logQueue;
> > >>>>>>
> > >>>>>> public void sendLogMessage(final LogMessage message) {
> > >>>>>> sendMessage(logQueue, message);
> > >>>>>> }
> > >>>>>>
> > >>>>>> private void sendMessage(final Queue queue, final
> CustomJmsMessage
> > >>>>>> message) {
> > >>>>>> try (final Connection connection =
> > >>>>>> connectionFactory.createConnection()) {
> > >>>>>> connection.start();
> > >>>>>>
> > >>>>>> final Session session = connection.createSession(true,
> > >>>>>> Session.AUTO_ACKNOWLEDGE);
> > >>>>>> final MessageProducer producer =
> > >>>> session.createProducer(queue);
> > >>>>>> final String serializedMessage =
> > >>>>>> CustomJsonProvider.toJson(message);
> > >>>>>> final Message jmsMessage =
> > >>>>>> session.createTextMessage(serializedMessage);
> > >>>>>>
> > >>>>>> // This enqueues messages successfully with both 8.0.0-M3
> > >> and
> > >>>>>> 8.0.0
> > >>>>>> producer.send(jmsMessage);
> > >>>>>> } catch (final Exception e) {
> > >>>>>> throw new RuntimeException("Caught exception from JMS
> when
> > >>>>>> sending a message", e);
> > >>>>>> }
> > >>>>>> }
> > >>>>>>
> > >>>>>> public TextMessage receiveLogMessage(final long
> > >>>> receiveTimeoutMillis) {
> > >>>>>> return receiveMessage(logQueue, receiveTimeoutMillis);
> > >>>>>> }
> > >>>>>>
> > >>>>>> private TextMessage receiveMessage(final Queue queue, final long
> > >>>>>> receiveTimeoutMillis) {
> > >>>>>> try (final Connection connection =
> > >>>>>> connectionFactory.createConnection()) {
> > >>>>>> connection.start();
> > >>>>>>
> > >>>>>> final Session session = connection.createSession(true,
> > >>>>>> Session.AUTO_ACKNOWLEDGE);
> > >>>>>> final MessageConsumer messageConsumer =
> > >>>>>> session.createConsumer(queue);
> > >>>>>> final Message jmsMessage =
> > >>>>>> messageConsumer.receive(receiveTimeoutMillis);
> > >>>>>>
> > >>>>>> // PROBLEM: jmsMessage is always null with 8.0.0. This
> was
> > >>>>>> working with 8.0.0-M3
> > >>>>>> if (jmsMessage == null) {
> > >>>>>> return null;
> > >>>>>> }
> > >>>>>>
> > >>>>>> return (TextMessage) jmsMessage;
> > >>>>>> } catch (final Exception e) {
> > >>>>>> throw new RuntimeException("Caught exception from JMS
> when
> > >>>>>> receiving a message", e);
> > >>>>>> }
> > >>>>>> }
> > >>>>>> }
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>>
> > >>
> > >>
> > >
> > > --
> > > Jonathan | exabrial@gmail.com
> > > Pessimists, see a jar as half empty. Optimists, in contrast, see it as
> > half
> > > full.
> > > Engineers, of course, understand the glass is twice as big as it needs
> to
> > > be.
> >
> >
>
--
Jonathan | exabrial@gmail.com
Pessimists, see a jar as half empty. Optimists, in contrast, see it as half
full.
Engineers, of course, understand the glass is twice as big as it needs to
be.
Re: Problem with MessageConsumers under TomEE 8.0.0
Posted by Jonathan Gallimore <jo...@gmail.com>.
That's interesting. We'll see if we can create some test cases for these
different scenarios.
Jon
On Thu, Oct 10, 2019 at 4:55 PM Ihsan Ecemis <mi...@gmail.com> wrote:
>
> Sorry I just had time to make these experiments.
>
> Having the following (setting transacted = false) did not fix the problem
> if we do not use try-with-resources:
>
> final Session session = connection.createSession(false,
> Session.AUTO_ACKNOWLEDGE);
>
>
> On the other hand, when we added transactionSupport=none to resources.xml,
> we could dequeue only 1 message, which is weird (at each server restart,
> we dequeue 1 message even though there are more message in the queue.
> Reverting the code back to try-with-resources makes the server dequeue all
> the messages)
>
>
> Hope that would help you guys find out where the problem is.
>
>
>
> > On Oct 1, 2019, at 16:42, Jonathan S. Fisher <ex...@gmail.com> wrote:
> >
> > Just curious, try this:
> >
> > final Session session = connection.createSession(false,
> > Session.AUTO_ACKNOWLEDGE);
> >
> > and/or try this:
> >
> > <Resource id="MyJmsConnectionFactory" type="javax.jms.ConnectionFactory">
> > ResourceAdapter = MyJmsResourceAdapter
> > transactionSupport=none
> > </Resource>
> >
> >
> >
> >
> > On Mon, Sep 30, 2019 at 8:53 PM Ihsan Ecemis <mi...@gmail.com> wrote:
> >
> >>
> >> Thanks for digging into this Jon (and pointing my attention toward that
> >> other paragraph in the Javadoc)
> >>
> >> And I highly appreciated the try-with-resources workaround.
> >>
> >>
> >>> On Sep 30, 2019, at 17:11, Jonathan Gallimore <
> >> jonathan.gallimore@gmail.com> wrote:
> >>>
> >>>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
> >>> states the following: There is no need to close the sessions,
> producers,
> >>> and consumers of a closed connection.
> >>>
> >>>> Here is the link:
> >>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
> <
> >>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
> >
> >>>
> >>> Good question. The changes in this area between 8.0.0-M3 and 8.0.0,
> are,
> >>> you'll be unsurprised to find out, around transaction handling. We need
> >> to
> >>> dig in a bit further.
> >>>
> >>> The Javadoc also states this: "Closing a connection causes any of its
> >>> sessions' transactions in progress to be rolled back. In the case
> where a
> >>> session's work is coordinated by an external transaction manager, a
> >>> session's commit and rollback methods are not used and the result of a
> >>> closed session's work is determined later by the transaction manager.
> >>> Closing a connection does NOT force an acknowledgment of
> >>> client-acknowledged sessions."
> >>>
> >>> A lot of this describes standalone usage, as opposed to usage in an
> >>> application server. The Connection object should actually be a proxy,
> and
> >>> calling close() shouldn't actually do anything. The sendMessage()
> method
> >> on
> >>> the CustomJmsService bean is transactional, and the connection should
> be
> >>> returned back to the pool once the method completes and the transaction
> >> is
> >>> committed.
> >>>
> >>> Thank you for the sample code, we should be able to debug through and
> see
> >>> what's going on. I should be able to do this over the next day - I'll
> >> post
> >>> my steps here in case you want to dig in and have a go yourself.
> >>>
> >>> Jon
> >>>
> >>> On Mon, Sep 30, 2019 at 6:32 PM Ihsan Ecemis <mi...@gmail.com>
> wrote:
> >>>
> >>>>
> >>>> Thank you very much for your suggestion Jon, adding
> >>>> connection.createSession(), session.createProducer(), and
> >>>> session.createConsumer() in the try-with-resources block make things
> >> work
> >>>> again!
> >>>>
> >>>>
> >>>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
> >>>> states the following: There is no need to close the sessions,
> >> producers,
> >>>> and consumers of a closed connection.
> >>>>
> >>>> Here is the link:
> >>>>
> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
> >> <
> >>>>
> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
> >>>>
> >>>>
> >>>> And our original code, that did not close/auto-close Session,
> >>>> MessageProducer, and MessageConsumer objects worked fine since at
> least
> >>>> TomEE 7.0.2 (our git history shows that we had such a piece of code
> >>>> implemented in March 2017)
> >>>>
> >>>>
> >>>>
> >>>>> On Sep 30, 2019, at 09:14, Jonathan Gallimore <
> >>>> jonathan.gallimore@gmail.com> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I'm wondering if this is because your send method isn't closing the
> >>>>> producer and the session? I did try your code and I saw the issue. I
> >>>> turned
> >>>>> it into an example, and added an Arquillian test:
> >>>>> https://github.com/apache/tomee/pull/578
> >>>>>
> >>>>> Note that I added the session, producer and consumer into the
> >>>>> try-with-resources block so they are auto-closed at the end of the
> >>>> method.
> >>>>>
> >>>>> Can you take a look and let me know what you think?
> >>>>>
> >>>>> Jon
> >>>>>
> >>>>> On Fri, Sep 27, 2019 at 6:05 PM Ihsan Ecemis <mi...@gmail.com>
> >> wrote:
> >>>>>
> >>>>>>
> >>>>>> Hello Everyone,
> >>>>>>
> >>>>>> We recently upgraded our staging environment from TomEE 8.0.0-M3 to
> >>>> 8.0.0
> >>>>>> and things started to break.
> >>>>>>
> >>>>>> After some troubleshooting, we realized that we cannot dequeue JMS
> >>>>>> messages from our ActiveMQ server using MessageConsumers.
> >>>>>>
> >>>>>>
> >>>>>> We dequeue messages in 2 different ways: We use MessageDriven beans
> >> for
> >>>>>> most of our queues. But with some others, we periodically poll by
> >>>> creating
> >>>>>> a MessageConsumer (please see the code below for that second case)
> >>>>>>
> >>>>>> MessageDriven beans work without any problems but we cannot receive
> >> any
> >>>>>> messages via MessageConsumers. That same backend code is working
> fine
> >>>> with
> >>>>>> 8.0.0-M3.
> >>>>>>
> >>>>>>
> >>>>>> We tested this with different versions of ActiveMQ server (5.15.6,
> >>>> 5.15.9,
> >>>>>> 5.15.10) but TomEE 8.0.0 did not work with any of them.
> >>>>>>
> >>>>>>
> >>>>>> Below is redacted code snippets showing where we have the problem.
> >>>>>>
> >>>>>> Any help will be greatly appreciated. Please let me know if you
> have
> >>>> any
> >>>>>> questions about our setup.
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Ihsan.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> @Stateless
> >>>>>> public class LogService {
> >>>>>>
> >>>>>> @EJB
> >>>>>> private CustomJmsService customJmsService;
> >>>>>>
> >>>>>> @javax.ejb.Schedule(second = "*/30", minute = "*", hour = "*")
> >>>>>> public void pollLogQueue() throws Exception {
> >>>>>> final TextMessage logMessage =
> >>>>>> customJmsService.receiveLogMessage(1000);
> >>>>>> if (logMessage != null) {
> >>>>>> persistLogMessages(logMessages);
> >>>>>> }
> >>>>>> }
> >>>>>> }
> >>>>>>
> >>>>>> @Stateless
> >>>>>> public class CustomJmsService {
> >>>>>>
> >>>>>> @Resource(name = "logQueue")
> >>>>>> private Queue logQueue;
> >>>>>>
> >>>>>> public void sendLogMessage(final LogMessage message) {
> >>>>>> sendMessage(logQueue, message);
> >>>>>> }
> >>>>>>
> >>>>>> private void sendMessage(final Queue queue, final CustomJmsMessage
> >>>>>> message) {
> >>>>>> try (final Connection connection =
> >>>>>> connectionFactory.createConnection()) {
> >>>>>> connection.start();
> >>>>>>
> >>>>>> final Session session = connection.createSession(true,
> >>>>>> Session.AUTO_ACKNOWLEDGE);
> >>>>>> final MessageProducer producer =
> >>>> session.createProducer(queue);
> >>>>>> final String serializedMessage =
> >>>>>> CustomJsonProvider.toJson(message);
> >>>>>> final Message jmsMessage =
> >>>>>> session.createTextMessage(serializedMessage);
> >>>>>>
> >>>>>> // This enqueues messages successfully with both 8.0.0-M3
> >> and
> >>>>>> 8.0.0
> >>>>>> producer.send(jmsMessage);
> >>>>>> } catch (final Exception e) {
> >>>>>> throw new RuntimeException("Caught exception from JMS when
> >>>>>> sending a message", e);
> >>>>>> }
> >>>>>> }
> >>>>>>
> >>>>>> public TextMessage receiveLogMessage(final long
> >>>> receiveTimeoutMillis) {
> >>>>>> return receiveMessage(logQueue, receiveTimeoutMillis);
> >>>>>> }
> >>>>>>
> >>>>>> private TextMessage receiveMessage(final Queue queue, final long
> >>>>>> receiveTimeoutMillis) {
> >>>>>> try (final Connection connection =
> >>>>>> connectionFactory.createConnection()) {
> >>>>>> connection.start();
> >>>>>>
> >>>>>> final Session session = connection.createSession(true,
> >>>>>> Session.AUTO_ACKNOWLEDGE);
> >>>>>> final MessageConsumer messageConsumer =
> >>>>>> session.createConsumer(queue);
> >>>>>> final Message jmsMessage =
> >>>>>> messageConsumer.receive(receiveTimeoutMillis);
> >>>>>>
> >>>>>> // PROBLEM: jmsMessage is always null with 8.0.0. This was
> >>>>>> working with 8.0.0-M3
> >>>>>> if (jmsMessage == null) {
> >>>>>> return null;
> >>>>>> }
> >>>>>>
> >>>>>> return (TextMessage) jmsMessage;
> >>>>>> } catch (final Exception e) {
> >>>>>> throw new RuntimeException("Caught exception from JMS when
> >>>>>> receiving a message", e);
> >>>>>> }
> >>>>>> }
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
> >
> > --
> > Jonathan | exabrial@gmail.com
> > Pessimists, see a jar as half empty. Optimists, in contrast, see it as
> half
> > full.
> > Engineers, of course, understand the glass is twice as big as it needs to
> > be.
>
>
Re: Problem with MessageConsumers under TomEE 8.0.0
Posted by Ihsan Ecemis <mi...@gmail.com>.
Sorry I just had time to make these experiments.
Having the following (setting transacted = false) did not fix the problem if we do not use try-with-resources:
final Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
On the other hand, when we added transactionSupport=none to resources.xml, we could dequeue only 1 message, which is weird (at each server restart, we dequeue 1 message even though there are more message in the queue. Reverting the code back to try-with-resources makes the server dequeue all the messages)
Hope that would help you guys find out where the problem is.
> On Oct 1, 2019, at 16:42, Jonathan S. Fisher <ex...@gmail.com> wrote:
>
> Just curious, try this:
>
> final Session session = connection.createSession(false,
> Session.AUTO_ACKNOWLEDGE);
>
> and/or try this:
>
> <Resource id="MyJmsConnectionFactory" type="javax.jms.ConnectionFactory">
> ResourceAdapter = MyJmsResourceAdapter
> transactionSupport=none
> </Resource>
>
>
>
>
> On Mon, Sep 30, 2019 at 8:53 PM Ihsan Ecemis <mi...@gmail.com> wrote:
>
>>
>> Thanks for digging into this Jon (and pointing my attention toward that
>> other paragraph in the Javadoc)
>>
>> And I highly appreciated the try-with-resources workaround.
>>
>>
>>> On Sep 30, 2019, at 17:11, Jonathan Gallimore <
>> jonathan.gallimore@gmail.com> wrote:
>>>
>>>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
>>> states the following: There is no need to close the sessions, producers,
>>> and consumers of a closed connection.
>>>
>>>> Here is the link:
>>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-- <
>>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
>>>
>>> Good question. The changes in this area between 8.0.0-M3 and 8.0.0, are,
>>> you'll be unsurprised to find out, around transaction handling. We need
>> to
>>> dig in a bit further.
>>>
>>> The Javadoc also states this: "Closing a connection causes any of its
>>> sessions' transactions in progress to be rolled back. In the case where a
>>> session's work is coordinated by an external transaction manager, a
>>> session's commit and rollback methods are not used and the result of a
>>> closed session's work is determined later by the transaction manager.
>>> Closing a connection does NOT force an acknowledgment of
>>> client-acknowledged sessions."
>>>
>>> A lot of this describes standalone usage, as opposed to usage in an
>>> application server. The Connection object should actually be a proxy, and
>>> calling close() shouldn't actually do anything. The sendMessage() method
>> on
>>> the CustomJmsService bean is transactional, and the connection should be
>>> returned back to the pool once the method completes and the transaction
>> is
>>> committed.
>>>
>>> Thank you for the sample code, we should be able to debug through and see
>>> what's going on. I should be able to do this over the next day - I'll
>> post
>>> my steps here in case you want to dig in and have a go yourself.
>>>
>>> Jon
>>>
>>> On Mon, Sep 30, 2019 at 6:32 PM Ihsan Ecemis <mi...@gmail.com> wrote:
>>>
>>>>
>>>> Thank you very much for your suggestion Jon, adding
>>>> connection.createSession(), session.createProducer(), and
>>>> session.createConsumer() in the try-with-resources block make things
>> work
>>>> again!
>>>>
>>>>
>>>> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
>>>> states the following: There is no need to close the sessions,
>> producers,
>>>> and consumers of a closed connection.
>>>>
>>>> Here is the link:
>>>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
>> <
>>>> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
>>>>
>>>>
>>>> And our original code, that did not close/auto-close Session,
>>>> MessageProducer, and MessageConsumer objects worked fine since at least
>>>> TomEE 7.0.2 (our git history shows that we had such a piece of code
>>>> implemented in March 2017)
>>>>
>>>>
>>>>
>>>>> On Sep 30, 2019, at 09:14, Jonathan Gallimore <
>>>> jonathan.gallimore@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm wondering if this is because your send method isn't closing the
>>>>> producer and the session? I did try your code and I saw the issue. I
>>>> turned
>>>>> it into an example, and added an Arquillian test:
>>>>> https://github.com/apache/tomee/pull/578
>>>>>
>>>>> Note that I added the session, producer and consumer into the
>>>>> try-with-resources block so they are auto-closed at the end of the
>>>> method.
>>>>>
>>>>> Can you take a look and let me know what you think?
>>>>>
>>>>> Jon
>>>>>
>>>>> On Fri, Sep 27, 2019 at 6:05 PM Ihsan Ecemis <mi...@gmail.com>
>> wrote:
>>>>>
>>>>>>
>>>>>> Hello Everyone,
>>>>>>
>>>>>> We recently upgraded our staging environment from TomEE 8.0.0-M3 to
>>>> 8.0.0
>>>>>> and things started to break.
>>>>>>
>>>>>> After some troubleshooting, we realized that we cannot dequeue JMS
>>>>>> messages from our ActiveMQ server using MessageConsumers.
>>>>>>
>>>>>>
>>>>>> We dequeue messages in 2 different ways: We use MessageDriven beans
>> for
>>>>>> most of our queues. But with some others, we periodically poll by
>>>> creating
>>>>>> a MessageConsumer (please see the code below for that second case)
>>>>>>
>>>>>> MessageDriven beans work without any problems but we cannot receive
>> any
>>>>>> messages via MessageConsumers. That same backend code is working fine
>>>> with
>>>>>> 8.0.0-M3.
>>>>>>
>>>>>>
>>>>>> We tested this with different versions of ActiveMQ server (5.15.6,
>>>> 5.15.9,
>>>>>> 5.15.10) but TomEE 8.0.0 did not work with any of them.
>>>>>>
>>>>>>
>>>>>> Below is redacted code snippets showing where we have the problem.
>>>>>>
>>>>>> Any help will be greatly appreciated. Please let me know if you have
>>>> any
>>>>>> questions about our setup.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Ihsan.
>>>>>>
>>>>>>
>>>>>>
>>>>>> @Stateless
>>>>>> public class LogService {
>>>>>>
>>>>>> @EJB
>>>>>> private CustomJmsService customJmsService;
>>>>>>
>>>>>> @javax.ejb.Schedule(second = "*/30", minute = "*", hour = "*")
>>>>>> public void pollLogQueue() throws Exception {
>>>>>> final TextMessage logMessage =
>>>>>> customJmsService.receiveLogMessage(1000);
>>>>>> if (logMessage != null) {
>>>>>> persistLogMessages(logMessages);
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> @Stateless
>>>>>> public class CustomJmsService {
>>>>>>
>>>>>> @Resource(name = "logQueue")
>>>>>> private Queue logQueue;
>>>>>>
>>>>>> public void sendLogMessage(final LogMessage message) {
>>>>>> sendMessage(logQueue, message);
>>>>>> }
>>>>>>
>>>>>> private void sendMessage(final Queue queue, final CustomJmsMessage
>>>>>> message) {
>>>>>> try (final Connection connection =
>>>>>> connectionFactory.createConnection()) {
>>>>>> connection.start();
>>>>>>
>>>>>> final Session session = connection.createSession(true,
>>>>>> Session.AUTO_ACKNOWLEDGE);
>>>>>> final MessageProducer producer =
>>>> session.createProducer(queue);
>>>>>> final String serializedMessage =
>>>>>> CustomJsonProvider.toJson(message);
>>>>>> final Message jmsMessage =
>>>>>> session.createTextMessage(serializedMessage);
>>>>>>
>>>>>> // This enqueues messages successfully with both 8.0.0-M3
>> and
>>>>>> 8.0.0
>>>>>> producer.send(jmsMessage);
>>>>>> } catch (final Exception e) {
>>>>>> throw new RuntimeException("Caught exception from JMS when
>>>>>> sending a message", e);
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> public TextMessage receiveLogMessage(final long
>>>> receiveTimeoutMillis) {
>>>>>> return receiveMessage(logQueue, receiveTimeoutMillis);
>>>>>> }
>>>>>>
>>>>>> private TextMessage receiveMessage(final Queue queue, final long
>>>>>> receiveTimeoutMillis) {
>>>>>> try (final Connection connection =
>>>>>> connectionFactory.createConnection()) {
>>>>>> connection.start();
>>>>>>
>>>>>> final Session session = connection.createSession(true,
>>>>>> Session.AUTO_ACKNOWLEDGE);
>>>>>> final MessageConsumer messageConsumer =
>>>>>> session.createConsumer(queue);
>>>>>> final Message jmsMessage =
>>>>>> messageConsumer.receive(receiveTimeoutMillis);
>>>>>>
>>>>>> // PROBLEM: jmsMessage is always null with 8.0.0. This was
>>>>>> working with 8.0.0-M3
>>>>>> if (jmsMessage == null) {
>>>>>> return null;
>>>>>> }
>>>>>>
>>>>>> return (TextMessage) jmsMessage;
>>>>>> } catch (final Exception e) {
>>>>>> throw new RuntimeException("Caught exception from JMS when
>>>>>> receiving a message", e);
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
>
> --
> Jonathan | exabrial@gmail.com
> Pessimists, see a jar as half empty. Optimists, in contrast, see it as half
> full.
> Engineers, of course, understand the glass is twice as big as it needs to
> be.
Re: Problem with MessageConsumers under TomEE 8.0.0
Posted by "Jonathan S. Fisher" <ex...@gmail.com>.
Just curious, try this:
final Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
and/or try this:
<Resource id="MyJmsConnectionFactory" type="javax.jms.ConnectionFactory">
ResourceAdapter = MyJmsResourceAdapter
transactionSupport=none
</Resource>
On Mon, Sep 30, 2019 at 8:53 PM Ihsan Ecemis <mi...@gmail.com> wrote:
>
> Thanks for digging into this Jon (and pointing my attention toward that
> other paragraph in the Javadoc)
>
> And I highly appreciated the try-with-resources workaround.
>
>
> > On Sep 30, 2019, at 17:11, Jonathan Gallimore <
> jonathan.gallimore@gmail.com> wrote:
> >
> >> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
> > states the following: There is no need to close the sessions, producers,
> > and consumers of a closed connection.
> >
> >> Here is the link:
> > https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-- <
> > https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
> >
> > Good question. The changes in this area between 8.0.0-M3 and 8.0.0, are,
> > you'll be unsurprised to find out, around transaction handling. We need
> to
> > dig in a bit further.
> >
> > The Javadoc also states this: "Closing a connection causes any of its
> > sessions' transactions in progress to be rolled back. In the case where a
> > session's work is coordinated by an external transaction manager, a
> > session's commit and rollback methods are not used and the result of a
> > closed session's work is determined later by the transaction manager.
> > Closing a connection does NOT force an acknowledgment of
> > client-acknowledged sessions."
> >
> > A lot of this describes standalone usage, as opposed to usage in an
> > application server. The Connection object should actually be a proxy, and
> > calling close() shouldn't actually do anything. The sendMessage() method
> on
> > the CustomJmsService bean is transactional, and the connection should be
> > returned back to the pool once the method completes and the transaction
> is
> > committed.
> >
> > Thank you for the sample code, we should be able to debug through and see
> > what's going on. I should be able to do this over the next day - I'll
> post
> > my steps here in case you want to dig in and have a go yourself.
> >
> > Jon
> >
> > On Mon, Sep 30, 2019 at 6:32 PM Ihsan Ecemis <mi...@gmail.com> wrote:
> >
> >>
> >> Thank you very much for your suggestion Jon, adding
> >> connection.createSession(), session.createProducer(), and
> >> session.createConsumer() in the try-with-resources block make things
> work
> >> again!
> >>
> >>
> >> Is this a bug with TomEE 8.0.0? Because Connection.close()’s Javadoc
> >> states the following: There is no need to close the sessions,
> producers,
> >> and consumers of a closed connection.
> >>
> >> Here is the link:
> >> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close--
> <
> >> https://docs.oracle.com/javaee/7/api/javax/jms/Connection.html#close-->
> >>
> >>
> >> And our original code, that did not close/auto-close Session,
> >> MessageProducer, and MessageConsumer objects worked fine since at least
> >> TomEE 7.0.2 (our git history shows that we had such a piece of code
> >> implemented in March 2017)
> >>
> >>
> >>
> >>> On Sep 30, 2019, at 09:14, Jonathan Gallimore <
> >> jonathan.gallimore@gmail.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I'm wondering if this is because your send method isn't closing the
> >>> producer and the session? I did try your code and I saw the issue. I
> >> turned
> >>> it into an example, and added an Arquillian test:
> >>> https://github.com/apache/tomee/pull/578
> >>>
> >>> Note that I added the session, producer and consumer into the
> >>> try-with-resources block so they are auto-closed at the end of the
> >> method.
> >>>
> >>> Can you take a look and let me know what you think?
> >>>
> >>> Jon
> >>>
> >>> On Fri, Sep 27, 2019 at 6:05 PM Ihsan Ecemis <mi...@gmail.com>
> wrote:
> >>>
> >>>>
> >>>> Hello Everyone,
> >>>>
> >>>> We recently upgraded our staging environment from TomEE 8.0.0-M3 to
> >> 8.0.0
> >>>> and things started to break.
> >>>>
> >>>> After some troubleshooting, we realized that we cannot dequeue JMS
> >>>> messages from our ActiveMQ server using MessageConsumers.
> >>>>
> >>>>
> >>>> We dequeue messages in 2 different ways: We use MessageDriven beans
> for
> >>>> most of our queues. But with some others, we periodically poll by
> >> creating
> >>>> a MessageConsumer (please see the code below for that second case)
> >>>>
> >>>> MessageDriven beans work without any problems but we cannot receive
> any
> >>>> messages via MessageConsumers. That same backend code is working fine
> >> with
> >>>> 8.0.0-M3.
> >>>>
> >>>>
> >>>> We tested this with different versions of ActiveMQ server (5.15.6,
> >> 5.15.9,
> >>>> 5.15.10) but TomEE 8.0.0 did not work with any of them.
> >>>>
> >>>>
> >>>> Below is redacted code snippets showing where we have the problem.
> >>>>
> >>>> Any help will be greatly appreciated. Please let me know if you have
> >> any
> >>>> questions about our setup.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Ihsan.
> >>>>
> >>>>
> >>>>
> >>>> @Stateless
> >>>> public class LogService {
> >>>>
> >>>> @EJB
> >>>> private CustomJmsService customJmsService;
> >>>>
> >>>> @javax.ejb.Schedule(second = "*/30", minute = "*", hour = "*")
> >>>> public void pollLogQueue() throws Exception {
> >>>> final TextMessage logMessage =
> >>>> customJmsService.receiveLogMessage(1000);
> >>>> if (logMessage != null) {
> >>>> persistLogMessages(logMessages);
> >>>> }
> >>>> }
> >>>> }
> >>>>
> >>>> @Stateless
> >>>> public class CustomJmsService {
> >>>>
> >>>> @Resource(name = "logQueue")
> >>>> private Queue logQueue;
> >>>>
> >>>> public void sendLogMessage(final LogMessage message) {
> >>>> sendMessage(logQueue, message);
> >>>> }
> >>>>
> >>>> private void sendMessage(final Queue queue, final CustomJmsMessage
> >>>> message) {
> >>>> try (final Connection connection =
> >>>> connectionFactory.createConnection()) {
> >>>> connection.start();
> >>>>
> >>>> final Session session = connection.createSession(true,
> >>>> Session.AUTO_ACKNOWLEDGE);
> >>>> final MessageProducer producer =
> >> session.createProducer(queue);
> >>>> final String serializedMessage =
> >>>> CustomJsonProvider.toJson(message);
> >>>> final Message jmsMessage =
> >>>> session.createTextMessage(serializedMessage);
> >>>>
> >>>> // This enqueues messages successfully with both 8.0.0-M3
> and
> >>>> 8.0.0
> >>>> producer.send(jmsMessage);
> >>>> } catch (final Exception e) {
> >>>> throw new RuntimeException("Caught exception from JMS when
> >>>> sending a message", e);
> >>>> }
> >>>> }
> >>>>
> >>>> public TextMessage receiveLogMessage(final long
> >> receiveTimeoutMillis) {
> >>>> return receiveMessage(logQueue, receiveTimeoutMillis);
> >>>> }
> >>>>
> >>>> private TextMessage receiveMessage(final Queue queue, final long
> >>>> receiveTimeoutMillis) {
> >>>> try (final Connection connection =
> >>>> connectionFactory.createConnection()) {
> >>>> connection.start();
> >>>>
> >>>> final Session session = connection.createSession(true,
> >>>> Session.AUTO_ACKNOWLEDGE);
> >>>> final MessageConsumer messageConsumer =
> >>>> session.createConsumer(queue);
> >>>> final Message jmsMessage =
> >>>> messageConsumer.receive(receiveTimeoutMillis);
> >>>>
> >>>> // PROBLEM: jmsMessage is always null with 8.0.0. This was
> >>>> working with 8.0.0-M3
> >>>> if (jmsMessage == null) {
> >>>> return null;
> >>>> }
> >>>>
> >>>> return (TextMessage) jmsMessage;
> >>>> } catch (final Exception e) {
> >>>> throw new RuntimeException("Caught exception from JMS when
> >>>> receiving a message", e);
> >>>> }
> >>>> }
> >>>> }
> >>>>
> >>>>
> >>>>
> >>
> >>
>
>
--
Jonathan | exabrial@gmail.com
Pessimists, see a jar as half empty. Optimists, in contrast, see it as half
full.
Engineers, of course, understand the glass is twice as big as it needs to
be.