[jira] [Commented] (BOOKKEEPER-37) Ordering of published messages is not preserved when doing asynchronous publication

["Ivan Kelly (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103500#comment-13103500 ] 

Ivan Kelly commented on BOOKKEEPER-37:
--------------------------------------

This solution effectively makes the asyncPublish synchronous. The problem here is that if you publish asynchronously twice to the same topic, there's nothing to stop the client from creating two channels to the server. BOOKKEEPER-55 is related. I think we need to create a class called TopicChannel or something, and put the queuing in that.

So the publish code would look something like.
{code}
        if (client.topic2Host.containsKey(topic)) {
            InetSocketAddress host = client.topic2Host.get(topic);
            TopicChannel tc = host2Channel.get(host);
            if (tc == null) {
                tc = new TopicChannel(host);
                TopicChannel oldtc = client.topic2Host.putIfAbsent(host, tc);
                if (oldtc != null) {
                    tc = oldtc;
                }
            }
            tc.doPublish(pubSubData);
        }
{code}
Then doPublish can either queue or send the data depending on whether it's connected or not. The only thing is I don't know how this will react with topic redirects :/

> Ordering of published messages is not preserved when doing asynchronous publication
> -----------------------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-37
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-37
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: hedwig-client
>            Reporter: Matthieu Morel
>         Attachments: BOOKKEEPER-37.patch, ClientMessageOrderingTest.java
>
>
> Symptoms: 
> - if a publisher sends messages asynchronously through the asyncPublish method, the ordering is not preserved in Hedwig
> Example: 
> - a publisher sends M1, M2, M3 by invoking the asyncPublish method
> The Hedwig broker may see these messages in the following order: M2, M1, M3
> A subscriber will also see messages as M2, M1, M3
> How to reproduce:
> - see attached test case: synchronous publishing preserves ordering, but asynchronous publishing does not.
> The cause of the problem:
> - my understanding is that this is due to asynchronous creation of multiple netty channels on the publisher side. There is no ordering since messages are not sent through the same channel. 
> Suggested solution:
> Some buffering on the publisher side would allow to reuse the same channel and maintain ordering.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira