[jira] [Comment Edited] (BOOKKEEPER-37) Ordering of published messages is not preserved when doing asynchronous publication

["Sijie Guo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528686#comment-13528686 ] 

Sijie Guo edited comment on BOOKKEEPER-37 at 12/11/12 5:39 AM:
---------------------------------------------------------------

rethinking of this issue, in order to preserve ordering of published messages, we need to handle it in both client-side and server-side which is caused by redirection.

Ivan's example in previous comment has indicated the problem happened in client-side. One possible thing is that we could use queuing mechanism like what we used in delivering messages in order to message handler. In step 5, we should not send P3 to HUB2 while we need to wait all messages published before P3 went to the right owner of hub server.

Even we could guarantee client side could handle this case, there is an issue in server side. Taking follow example:

1. there are N publish request in hub server HUB1 in queue.
2. At time T, topic is released and HUB2 is chose as the new owner.
3. 1 - K requests are redirected.
4. HUB2 failed.
5. K+1 request executed in HUB1, which chose HUB1 as the owner.
6. K+1 ~ N would be executed.
7. 1 - K requests would be redirected back to HUB1 again, which are out-of-order messages after K+1 ~ N.

In order to guarantee order, we should ask HUB server to redirect all pending requests when releasing the ownership of a topic. otherwise, we have chance to get out-of-order messages as described above.

I am not sure if I missed other cases for it. I would try to work on generating on a patch for it. Did you guys think it is a blocker for 4.2.0 or not?




                
      was (Author: hustlmsp):
    rethinking of this issue, in order to preserve ordering of published messages, we need to handle it in both client-side and server-side by redirection.

Ivan's example in previous comment has indicated the problem happened in client-side. One possible thing is that we could use queuing mechanism like what we used in delivering messages in order to message handler. In step 5, we should not send P3 to HUB2 while we need to wait all messages published before P3 went to the right owner of hub server.

Even we could guarantee client side could handle this case, there is an issue in server side. Taking follow example:

1. there are N publish request in hub server HUB1 in queue.
2. At time T, topic is released and HUB2 is chose as the new owner.
3. 1 - K requests are redirected.
4. HUB2 failed.
5. K+1 request executed in HUB1, which chose HUB1 as the owner.
6. K+1 ~ N would be executed.
7. 1 - K requests would be redirected back to HUB1 again, which are out-of-order messages after K+1 ~ N.

In order to guarantee order, we should ask HUB server to redirect all pending requests when releasing the ownership of a topic. otherwise, we have chance to get out-of-order messages as described above.

I am not sure if I missed other cases for it. I would try to work on generating on a patch for it. Did you guys think it is a blocker for 4.2.0 or not?




                  
> Ordering of published messages is not preserved when doing asynchronous publication
> -----------------------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-37
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-37
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: hedwig-client
>    Affects Versions: 4.0.0
>            Reporter: Matthieu Morel
>            Assignee: Matthieu Morel
>             Fix For: 4.2.0
>
>         Attachments: BOOKKEEPER-37.patch, ClientMessageOrderingTest.java
>
>
> Symptoms: 
> - if a publisher sends messages asynchronously through the asyncPublish method, the ordering is not preserved in Hedwig
> Example: 
> - a publisher sends M1, M2, M3 by invoking the asyncPublish method
> The Hedwig broker may see these messages in the following order: M2, M1, M3
> A subscriber will also see messages as M2, M1, M3
> How to reproduce:
> - see attached test case: synchronous publishing preserves ordering, but asynchronous publishing does not.
> The cause of the problem:
> - my understanding is that this is due to asynchronous creation of multiple netty channels on the publisher side. There is no ordering since messages are not sent through the same channel. 
> Suggested solution:
> Some buffering on the publisher side would allow to reuse the same channel and maintain ordering.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira