You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org> on 2009/02/24 16:00:02 UTC
[jira] Created: (QPID-1677) Provide the ability to publish
synchronously
Provide the ability to publish synchronously
--------------------------------------------
Key: QPID-1677
URL: https://issues.apache.org/jira/browse/QPID-1677
Project: Qpid
Issue Type: Improvement
Components: Java Client
Affects Versions: M4
Reporter: Rajith Attapattu
Assignee: Rajith Attapattu
Fix For: M5
In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
(sync_persistent is deprecated but supported for a few releases with a warning printed if used).
In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Resolved: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajith Attapattu resolved QPID-1677.
------------------------------------
Resolution: Fixed
I am resolving this as there are no further comments or concerns regarding this.
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Attachments: QPID_1677.patch
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajith Attapattu updated QPID-1677:
-----------------------------------
Attachment: QPID_1677.patch
This patch contains the code for the fucntionality described here.
This also contains some code for QPID-1106.
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Fix For: M5
>
> Attachments: QPID_1677.patch
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676330#action_12676330 ]
Rajith Attapattu commented on QPID-1677:
----------------------------------------
I realized my argument about exactly-once semantics are wrong here (Thanks Gordon for pointing it out).
There is always the possibility of the last message being duplicated as the broker could crash before us sending the ack for the last message.
Unless we build an idempotency barrier on the JMS client it is not possible to provide exactly-once semantics.
In fact the JMS spec is fine with the last message being redelivered.
"When a client uses the AUTO_ACKNOWLEDGE mode, it is not in direct
control of message acknowledgment. Since such clients cannot know for
certain if a particular message has been acknowledged, they must be prepared
for redelivery of the last consumed message. This can be caused by the client
completing its work just prior to a failure that prevents the message
acknowledgment from occurring. Only a session's last consumed message is
subject to this ambiguity."
Also the spec does not say anything about the publish side.
The sync on publish could also duplicate the last message as the broker could receive the message replicate it across the cluster and then crash before acknowleding.
So while syc_publish will reduce the duplicate messages to 1 for a publisher it doesn't eliminate it.
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Fix For: M5
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699736#action_12699736 ]
Rajith Attapattu commented on QPID-1677:
----------------------------------------
I understand that in 0-8/0-9 you can't do a sync point. But I had it there thinking about the next AMQP version.
I don't know which way we will go when we add 1.0 support. But since it's only a small amount of code I just added it anyways.
You would need to have sync publishing for persistent messages if you want to have script JMS compliance. So this is more a JMS thingy than a version specific thing.
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Attachments: QPID_1677.patch
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699184#action_12699184 ]
Martin Ritchie commented on QPID-1677:
--------------------------------------
Rajith, why is the setPublishMode() code in the abstract BasicMessageProducer when it is used for 0-10 producing?
Think this code needs to be moved in to the protocol specific class.
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Attachments: QPID_1677.patch
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rob Godfrey (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676334#action_12676334 ]
Rob Godfrey commented on QPID-1677:
-----------------------------------
The following in the JMS spec certainly implies that send() should be synchronous in AMQP terms:
"If a failure occurs between the time a client commits its work on a Session and
the commit method returns, the client cannot determine if the transaction was
committed or rolled back. The same ambiguity exists when a failure occurs
between the non-transactional send of a PERSISTENT message and the return
from the sending method."
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Fix For: M5
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676358#action_12676358 ]
Rajith Attapattu commented on QPID-1677:
----------------------------------------
Appologies for not making my comment clear.
Certainly agree that the JMS spec implies that send() should be synchronous in AMQP terms for PERSISTENT messages and we do have the system/connection URL prop sync_persistence for this.
What I intended to say was about transient messages and that synchronous publish for transient messages could help in avoiding duplicates allthough it doesn't eliminate it completely during a broker failover. For that you would need an idempotency barrier built into the client.
In the description I incorrectly used the definition for AUTO_ACK as a justification to introduce the sync_publish. It is in that context I mentioned "Also the spec does not say anything about the publish side". If an idempotency barrier is built in the client then the publish side does not have to synchronously publish to achieve exactly-once either. So my justification for sync_publish was wrong.
On a separate note, JMS does not require to provide exactly-once semantics, it allows the last consumed message to be duplicated.
But in our current implementation there could be two duplicate messages for a single producer/consumer pair even when using synchronous publishing and synchronous acking on the consumer side.
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Fix For: M5
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie updated QPID-1677:
---------------------------------
Fix Version/s: (was: M5)
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Attachments: QPID_1677.patch
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682355#action_12682355 ]
Rajith Attapattu commented on QPID-1677:
----------------------------------------
The solution has been committed in Qpid trunk at rev 747962.
Using sync_publish={persistent|all} only the last message published could be subject to ambiguity.
sync_persistent is retained for legacy support and a warning (mentioning it is deprecated and asking to use sync_publish) is printed if used.
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Attachments: QPID_1677.patch
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-1677) Provide the ability to publish
synchronously
Posted by "Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajith Attapattu updated QPID-1677:
-----------------------------------
Status: Ready To Review (was: In Progress)
> Provide the ability to publish synchronously
> --------------------------------------------
>
> Key: QPID-1677
> URL: https://issues.apache.org/jira/browse/QPID-1677
> Project: Qpid
> Issue Type: Improvement
> Components: Java Client
> Affects Versions: M4
> Reporter: Rajith Attapattu
> Assignee: Rajith Attapattu
> Attachments: QPID_1677.patch
>
>
> In order for the producer to not replay the messages again (during failover) and prevent duplicates,we need a way for the JMS client to sync on every publish.
> To faciliate this, I have propose a System/Connection URLproperty called sync_publish={persistent|all} which supercedes sync_persistent.
> (sync_persistent is deprecated but supported for a few releases with a warning printed if used).
> In order to provide exactly-once semantics between a single JMS publisher-consumer pair (when using AUTO_ACK on the consumer) we need to publish synchronously and ack synchronously (QPID-1106).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org