[jira] [Comment Edited] (RATIS-477) Improve LogStateMachine.processReadRequest()

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/RATIS-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16756683#comment-16756683 ] 

Josh Elser edited comment on RATIS-477 at 1/30/19 11:45 PM:
------------------------------------------------------------

{quote}we can use this LogID (RAFT) as our start log ID to avoid iterating through all logs
{quote}
You're right that we're returning back the RaftLog index via the append(), but I think that's a bug right now. For example, take this sequence:
 # write 'a', returns id=1
 # write ['b', 'c', 'd'], returns id=2

If you asked for id=3 (trying to read the value 'c'), we would read off the end of the RaftLog. I think for all DML operations, we want to use our own index that is disjoint from the RaftLog index. I think this gives a much easier client implementation and user experience, and gives us the chance to hide this complexity server-side.

Specifically, I think the above should return:
 # write 'a', returns id=1
 # write ['b', 'c', 'd'], returns id=[2,3,4]

For example, I bet we could maintain a sorted-map of recordId->RaftLog ID and then binary-search over it to make the RaftLog accesses fast.

Does this make sense? I'm not sure if my idea for design is clear. (totally possible I misunderstood your point, too :))


was (Author: elserj):
{quote}we can use this LogID (RAFT) as our start log ID to avoid iterating through all logs
{quote}
You're right that we're returning back the RaftLog index via the append(), but I think that's a bug right now. For example, take this sequence:
 # write 'a', returns id=1
 # write ['b', 'c', 'd'], returns id=2

If you asked for id=3 (trying to read the value 'c'), we would read off the end of the RaftLog. I think for all DML operations, we want to use our own index that is disjoint from the RaftLog index. I think this gives a much easier client implementation and user experience, and gives us the chance to hide this complexity server-side.

For example, I bet we could maintain a sorted-map of recordId->RaftLog ID and then binary-search over it to make the RaftLog accesses fast.

Does this make sense? I'm not sure if my idea for design is clear.

> Improve LogStateMachine.processReadRequest()
> --------------------------------------------
>
>                 Key: RATIS-477
>                 URL: https://issues.apache.org/jira/browse/RATIS-477
>             Project: Ratis
>          Issue Type: Improvement
>          Components: LogService
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Major
>             Fix For: 0.4.0
>
>
> [~vrodionov] and [~rajeshbabu] were asking about this in RATIS-470. Can we improve the logic that reads through the "head" of the RaftLog to find the "LogService offset"?
> Vlad suggested:
> {quote}
> Josh, can we simplify this a little bit? Client gets Raft log ID one every append, for multi-get calls we can use this LogID (RAFT) as our start log ID to avoid iterating through all logs, then you can unwind log entries by using new logic, see above:
>  int numRecordsInAppend = append.getDataCount();
> There is no need to iterate from the very beginning
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)