You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by john cheng <zq...@gmail.com> on 2016/01/22 01:46:56 UTC

question about NetworkClient inFlightRequests operation

I'm reading new client design of version 0.9. and I has a question of
inFlightRequests in and out.
Here is the basic flow :

When Sender send a ClientRequest to NetworkClient, it add to
inFlightRequests indicator in-flight requests

```
    private void doSend(ClientRequest request, long now) {
        this.inFlightRequests.add(request);
        selector.send(request.request());
    }
```

the inFlightRequests map node to deque. the new request add as first
element of deque

```
    public void add(ClientRequest request) {
        Deque<ClientRequest> reqs =
this.requests.get(request.request().destination());
        if (reqs == null) {
            reqs = new ArrayDeque<>();
            this.requests.put(request.request().destination(), reqs);
        }
        reqs.addFirst(request);
    }
```

then poll happen on client and then selector, after success send this
ClientRequest, the send will add to selector's completedSends

```
    private void handleCompletedSends(List<ClientResponse> responses, long
now) {
        // if no response is expected then when the send is completed,
return it
        for (Send send : this.selector.completedSends()) {
            ClientRequest request =
this.inFlightRequests.lastSent(send.destination());
            if (!request.expectResponse()) {
                this.inFlightRequests.completeLastSent(send.destination());
                responses.add(new ClientResponse(request, now, false,
null));
            }
        }
    }
```

if this request does't need response, the ClientRequest will remove from
inFlightRequest

```
    public ClientRequest completeLastSent(String node) {
        return requestQueue(node).pollFirst();
    }
```

----

I'm curios why poll First? A scene like this: after the first ClientRequest
sended out success,
and not yet execute to handleCompletedSends, another ClientRequest coming,
and the new request addFirst to deque.
then pollFirst execute, as first element of deque now become to the new
request, pollFirst will delete the new one, not old one.

```
CR1->inFlightRequests  |  CR1 send success  | CR2->inFlightRequests  |
completeSends  |  pollFirst

first    last                                 first    last
-------------                                 -------------
CR1                                           CR2 CR1
 CR1           poll CR2, but CR2 is just come in!
-------------                                 -------------
```

I has also check `NetworkClient.send->canSendRequest ->
inFlightRequests.canSendMore(node) ->
queue.peekFirst().request().completed()`
only the first element of deque finish, then new request can send to the
same node. but the condition of completed
by `ByteBufferSend` is `remaining <= 0 && !pending`. which means If Send
sended success to server, it's completed!

Am I missig something(are there any other limitation)? Can some on point
out. Tks.