Heads up: pushed write path restructuring

[Todd Lipcon <to...@cloudera.com>]

Hey all,

I just pushed a patch which restructures the write path for batched
operations in a moderately serious way[1]. It's been through several rounds
of review and a good amount of stress testing, but I wouldn't be super
surprised if there were some bug which created a flaky test.

I'll be watching out on the flaky dashboard the next couple days, but if
you see any funky looking test failures please ping me to take a look. If
anything appears to regress, we can revert and re-land after debugging.

-Todd

[1]
https://github.com/apache/kudu/commit/4568f2a4cbd999d5a2ab64d285205b8d30997339

-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Heads up: pushed write path restructuring

[Todd Lipcon <to...@cloudera.com>]

Think I got a handle on this:

The updates in LinkedListTest specifically directly "trail" the inserts -
i.e we insert rows, put the row keys onto a queue, and then the updater
thread immediately updates the same rows which were just inserted.

This means that most of the updates should hit MRS, rather thane one of the
DRS. Prior to my patch, we would check MRS for updates, and only if it
missed there, go and consult DRS. Now, we do things in the opposite
direction, and check all the MRS for the row before we try MRS (only if we
didn't find it anywhere).

This is probably a common enough use case (updating recently-inserted data)
that it's worth preserving the optimized path. I'll work on how to keep
that optimization without hurting code clarity too much.

-Todd

On Thu, Apr 6, 2017 at 10:46 AM, Todd Lipcon <to...@cloudera.com> wrote:

> Seeing some issues with write ops timing out in linked_list-test:
> http://dist-test.cloudera.org:8080/test_drilldown?test_name=linked_
> list-test
>
> It seems like the timeouts are from the updater thread, so it's possible
> the patch regressed performance for UPDATEs -- the benchmarking focused on
> INSERT workload.
>
> I'll investigate this today and if it seems like there's a big regression,
> will either fix or revert.
>
> -Todd
>
>
> On Wed, Apr 5, 2017 at 10:21 PM, Todd Lipcon <to...@cloudera.com> wrote:
>
>> Hey all,
>>
>> I just pushed a patch which restructures the write path for batched
>> operations in a moderately serious way[1]. It's been through several rounds
>> of review and a good amount of stress testing, but I wouldn't be super
>> surprised if there were some bug which created a flaky test.
>>
>> I'll be watching out on the flaky dashboard the next couple days, but if
>> you see any funky looking test failures please ping me to take a look. If
>> anything appears to regress, we can revert and re-land after debugging.
>>
>> -Todd
>>
>> [1] https://github.com/apache/kudu/commit/4568f2a4cbd999d5a2
>> ab64d285205b8d30997339
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Heads up: pushed write path restructuring

[Todd Lipcon <to...@cloudera.com>]

Seeing some issues with write ops timing out in linked_list-test:
http://dist-test.cloudera.org:8080/test_drilldown?test_name=linked_list-test

It seems like the timeouts are from the updater thread, so it's possible
the patch regressed performance for UPDATEs -- the benchmarking focused on
INSERT workload.

I'll investigate this today and if it seems like there's a big regression,
will either fix or revert.

-Todd


On Wed, Apr 5, 2017 at 10:21 PM, Todd Lipcon <to...@cloudera.com> wrote:

> Hey all,
>
> I just pushed a patch which restructures the write path for batched
> operations in a moderately serious way[1]. It's been through several rounds
> of review and a good amount of stress testing, but I wouldn't be super
> surprised if there were some bug which created a flaky test.
>
> I'll be watching out on the flaky dashboard the next couple days, but if
> you see any funky looking test failures please ping me to take a look. If
> anything appears to regress, we can revert and re-land after debugging.
>
> -Todd
>
> [1] https://github.com/apache/kudu/commit/4568f2a4cbd999d5a2ab64d285205b
> 8d30997339
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Todd Lipcon
Software Engineer, Cloudera