You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org> on 2017/03/03 22:44:58 UTC

[kudu-CR] WIP: KUDU-1905 - Allow reinserts on pk only tables 

David Ribeiro Alves has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/6258

Change subject: WIP: KUDU-1905 - Allow reinserts on pk only tables
......................................................................

WIP: KUDU-1905 - Allow reinserts on pk only tables

Doing a reinsert to a table that as only primary key columns
results in an empty change list. We're currently crashing whenever
we see a empty changelist that is not a delete.

The fix is just to allow empty changelists for reinserts.
This also adds a new flavor of fuzz tests to fuzz-itest.cc
that only have pk-only operations, as well as a directed
regression test that would trigger the problem deterministically.

WIP cuz still running dist-tests on this.

Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
---
M src/kudu/common/row_changelist.cc
M src/kudu/integration-tests/fuzz-itest.cc
2 files changed, 103 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/6258/1
-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................


Patch Set 4: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/6258/2/src/kudu/integration-tests/fuzz-itest.cc
File src/kudu/integration-tests/fuzz-itest.cc:

PS2, Line 446:       case TEST_INSERT:
             :         if (exists[row_key]) continue;
             :         ops->push_back({TEST_INSERT, row_key});
             :         exists[row_key] = true;
             :         ops_pending = true;
             :         data_in_mrs = true;
             :         break;
             :       case TEST_INSERT_PK_ONLY:
             :         if (exists[row_key]) continue;
             :         ops->push_back({TEST_INSERT_PK_ONLY, row_key});
             :         exists[row_key] = true;
             :         ops_pending = true;
             :         data_in_mrs = true;
             :  
> you can combine these two cases by doing ops->push_back({r, row_key}) right
Done


-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................


Patch Set 2:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/6258/1//COMMIT_MSG
Commit Message:

PS1, Line 9: as
has


http://gerrit.cloudera.org:8080/#/c/6258/1/src/kudu/common/row_changelist.cc
File src/kudu/common/row_changelist.cc:

Line 163:   // Note that REINSERTs might have empty changelists when reinserting a row on a tablet that
should we just say is_update()?


http://gerrit.cloudera.org:8080/#/c/6258/2/src/kudu/integration-tests/fuzz-itest.cc
File src/kudu/integration-tests/fuzz-itest.cc:

Line 237:         // For "upsert PK only", we expect the row to keep its old value
nit: missing word 'if'


PS2, Line 242: c
typo


PS2, Line 406: enum TestOpSets {
             :   ALL,
             :   PK_ONLY
             : };
what about doing something like:

vector<TestOpType> kAllOps = { TEST_INSERT, TEST_UPDATE, TEST_UPSERT, TEST_INSERT_PK_ONLY, ...}
vector<TestOpType> kPkOnlyOps = { TEST_INSERT_PK_ONLY, ...}

or somesuch? then the PickOpAtRandom can just pick a random one based on op_types.size() instead of the kind of funny "retry until we get one which matches" loop


PS2, Line 446:       case TEST_INSERT:
             :         if (exists[row_key]) continue;
             :         ops->push_back({TEST_INSERT, row_key});
             :         exists[row_key] = true;
             :         ops_pending = true;
             :         data_in_mrs = true;
             :         break;
             :       case TEST_INSERT_PK_ONLY:
             :         if (exists[row_key]) continue;
             :         ops->push_back({TEST_INSERT_PK_ONLY, row_key});
             :         exists[row_key] = true;
             :         ops_pending = true;
             :         data_in_mrs = true;
             :  
you can combine these two cases by doing ops->push_back({r, row_key}) right?


-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/6258

to look at the new patch set (#3).

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................

KUDU-1905 - Allow reinserts on pk only tables

Doing a reinsert to a table that has only primary key columns
results in an empty change list. We're currently crashing whenever
we see a empty changelist that is not a delete.

The fix is just to allow empty changelists for reinserts.
This also adds a new flavor of fuzz tests to fuzz-itest.cc
that only have pk-only operations, as well as a directed
regression test that would trigger the problem deterministically.

Ran fuzz-itest in dist-tests with the new tests and the following
command:
KUDU_ALLOW_SLOW_TESTS=1 build-support/dist_test.py --collect-tmpdir \
loop -n 1000 build/debug/bin/fuzz-itest --gtest_repeat=10 \
--gtest_break_on_failure

Tests passed 1000/1000. Results:
http://dist-test.cloudera.org//job?job_id=david.alves.1488580839.22665

Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
---
M src/kudu/common/row_changelist-test.cc
M src/kudu/common/row_changelist.cc
M src/kudu/integration-tests/fuzz-itest.cc
3 files changed, 123 insertions(+), 30 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/6258/3
-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/6258

to look at the new patch set (#2).

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................

KUDU-1905 - Allow reinserts on pk only tables

Doing a reinsert to a table that as only primary key columns
results in an empty change list. We're currently crashing whenever
we see a empty changelist that is not a delete.

The fix is just to allow empty changelists for reinserts.
This also adds a new flavor of fuzz tests to fuzz-itest.cc
that only have pk-only operations, as well as a directed
regression test that would trigger the problem deterministically.

Ran fuzz-itest in dist-tests with the new tests and the following
command:
KUDU_ALLOW_SLOW_TESTS=1 build-support/dist_test.py --collect-tmpdir \
loop -n 1000 build/debug/bin/fuzz-itest --gtest_repeat=10 \
--gtest_break_on_failure

Tests passed 1000/1000. Results:
http://dist-test.cloudera.org//job?job_id=david.alves.1488580839.22665

Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
---
M src/kudu/common/row_changelist-test.cc
M src/kudu/common/row_changelist.cc
M src/kudu/integration-tests/fuzz-itest.cc
3 files changed, 105 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/6258/2
-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged.

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................


KUDU-1905 - Allow reinserts on pk only tables

Doing a reinsert to a table that has only primary key columns
results in an empty change list. We're currently crashing whenever
we see a empty changelist that is not a delete.

The fix is just to allow empty changelists for reinserts.
This also adds a new flavor of fuzz tests to fuzz-itest.cc
that only have pk-only operations, as well as a directed
regression test that would trigger the problem deterministically.

Ran fuzz-itest in dist-tests with the new tests and the following
command:
KUDU_ALLOW_SLOW_TESTS=1 build-support/dist_test.py --collect-tmpdir \
loop -n 1000 build/debug/bin/fuzz-itest --gtest_repeat=10 \
--gtest_break_on_failure

Tests passed 1000/1000. Results:
http://dist-test.cloudera.org//job?job_id=david.alves.1488580839.22665

Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Reviewed-on: http://gerrit.cloudera.org:8080/6258
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon <to...@apache.org>
---
M src/kudu/common/row_changelist-test.cc
M src/kudu/common/row_changelist.cc
M src/kudu/integration-tests/fuzz-itest.cc
3 files changed, 118 insertions(+), 31 deletions(-)

Approvals:
  Todd Lipcon: Looks good to me, approved
  Kudu Jenkins: Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................


Patch Set 3:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/6258/1//COMMIT_MSG
Commit Message:

PS1, Line 9: ha
> has
Done


http://gerrit.cloudera.org:8080/#/c/6258/1/src/kudu/common/row_changelist.cc
File src/kudu/common/row_changelist.cc:

Line 163:   // Note that REINSERTs might have empty changelists when reinserting a row on a tablet that
> should we just say is_update()?
Done


http://gerrit.cloudera.org:8080/#/c/6258/2/src/kudu/integration-tests/fuzz-itest.cc
File src/kudu/integration-tests/fuzz-itest.cc:

Line 237:   ExpectedKeyValueRow InsertOrUpsertRow(int key, int val,
> nit: missing word 'if'
Done


PS2, Line 242: =
> typo
Done


PS2, Line 406: 
             :     string final_error;
             :     if (!errors.empty()) {
             :   
> what about doing something like:
Done


-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1905 - Allow reinserts on pk only tables

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/6258

to look at the new patch set (#4).

Change subject: KUDU-1905 - Allow reinserts on pk only tables
......................................................................

KUDU-1905 - Allow reinserts on pk only tables

Doing a reinsert to a table that has only primary key columns
results in an empty change list. We're currently crashing whenever
we see a empty changelist that is not a delete.

The fix is just to allow empty changelists for reinserts.
This also adds a new flavor of fuzz tests to fuzz-itest.cc
that only have pk-only operations, as well as a directed
regression test that would trigger the problem deterministically.

Ran fuzz-itest in dist-tests with the new tests and the following
command:
KUDU_ALLOW_SLOW_TESTS=1 build-support/dist_test.py --collect-tmpdir \
loop -n 1000 build/debug/bin/fuzz-itest --gtest_repeat=10 \
--gtest_break_on_failure

Tests passed 1000/1000. Results:
http://dist-test.cloudera.org//job?job_id=david.alves.1488580839.22665

Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
---
M src/kudu/common/row_changelist-test.cc
M src/kudu/common/row_changelist.cc
M src/kudu/integration-tests/fuzz-itest.cc
3 files changed, 118 insertions(+), 31 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/6258/4
-- 
To view, visit http://gerrit.cloudera.org:8080/6258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ce03378c7b97fac8ad8cb7783dec4a1b0277344
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>