You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Adar Dembo (Code Review)" <ge...@cloudera.org> on 2018/11/08 20:28:18 UTC

[kudu-CR] KUDU-2378: fix another aligned load crash

Hello Alexey Serbin, Grant Henke, Todd Lipcon,

I'd like you to do a code review. Please visit

    http://gerrit.cloudera.org:8080/11911

to review the following change.


Change subject: KUDU-2378: fix another aligned load crash
......................................................................

KUDU-2378: fix another aligned load crash

The TestKuduBackup.testRandomBackupAndRestore test from the kudu-backup
project would crash a TSAN master from time to time, with a stack trace
ending in Variant::Reset. After some debugging, this turned out to be a
holdover from KUDU-2378, where clang emitted a movaps (Aligned Load)
instruction on a value that wasn't aligned. The holdover was missed
because it was a static_cast (the original search covered
reinterpret_casts); I did a search for other bad casts but found none.

Why TSAN? Probably because it's the only build to use libc++, whose
std::string uses the Small String Optimization to embed the data into the
string itself, thus raising the possibility of the string's data being
unaligned. By comparison, libstdc++'s COW std::string uses a separate
allocation for string data, which is likely always 16-byte aligned.

The patch includes a unit test that isn't totally necessary, but was a
useful way to repro the crash in relative isolation.

Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
---
M src/kudu/common/types.h
M src/kudu/master/master-test.cc
2 files changed, 21 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/11911/1
-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11911/2/src/kudu/common/types.h
File src/kudu/common/types.h:

http://gerrit.cloudera.org:8080/#/c/11911/2/src/kudu/common/types.h@702
PS2, Line 702:         numeric_.u64 = *static_cast<const uint64_t *>(value);
mind fixing all these others, too, so that whenever some poor soul tries to port kudu to Alpha or something, it won't crash?



-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 08 Nov 2018 22:18:35 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Patch Set 4: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 09 Nov 2018 02:07:31 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Patch Set 1: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 08 Nov 2018 21:27:00 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Grant Henke, Todd Lipcon, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11911

to look at the new patch set (#2).

Change subject: KUDU-2378: fix another aligned load crash
......................................................................

KUDU-2378: fix another aligned load crash

The TestKuduBackup.testRandomBackupAndRestore test from the kudu-backup
project would crash a TSAN master from time to time, with a stack trace
ending in Variant::Reset. After some debugging, this turned out to be a
holdover from KUDU-2378, where clang emitted a movaps (Aligned Load)
instruction on a value that wasn't aligned. The holdover was missed
because it was a static_cast (the original search covered
reinterpret_casts); I did a search for other bad casts but found none.

Why TSAN? Probably because it's the only build to use libc++, whose
std::string uses the Small String Optimization to embed the data into the
string itself, thus raising the possibility of the string's data being
unaligned. By comparison, libstdc++'s COW std::string uses a separate
allocation for string data, which is likely always 16-byte aligned.

The patch includes a unit test that would repro the crash 100% of the time
when built for TSAN.

Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
---
M src/kudu/common/types.h
M src/kudu/common/wire_protocol-test.cc
2 files changed, 10 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/11911/2
-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has removed Kudu Jenkins from this change.  ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Removed reviewer Kudu Jenkins with the following votes:

* Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11911/2/src/kudu/common/types.h
File src/kudu/common/types.h:

http://gerrit.cloudera.org:8080/#/c/11911/2/src/kudu/common/types.h@702
PS2, Line 702:         numeric_.u64 = *static_cast<const uint64_t *>(value);
> mind fixing all these others, too, so that whenever some poor soul tries to
Todd and I talked offline. He started on a patch to fix all aligned loads, but decided it wasn't a high priority fix and shelved it. He gave his blessing to this patch which just fixes the int128 case, which is actually critical for x86.



-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 3
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 08 Nov 2018 23:40:33 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Grant Henke, Todd Lipcon, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11911

to look at the new patch set (#3).

Change subject: KUDU-2378: fix another aligned load crash
......................................................................

KUDU-2378: fix another aligned load crash

The TestKuduBackup.testRandomBackupAndRestore test from the kudu-backup
project would crash a TSAN master from time to time, with a stack trace
ending in Variant::Reset. After some debugging, this turned out to be a
holdover from KUDU-2378, where clang emitted a movaps (Aligned Load)
instruction on a value that wasn't aligned. The holdover was missed
because it was a static_cast (the original search covered
reinterpret_casts); I did a search for other bad casts but found none.

Why TSAN? Probably because it's the only build to use libc++, whose
std::string uses the Small String Optimization to embed the data into the
string itself, thus raising the possibility of the string's data being
unaligned. By comparison, libstdc++'s COW std::string uses a separate
allocation for string data, which is likely always 16-byte aligned.

The patch includes a unit test that would repro the crash 100% of the time
when built for TSAN.

Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
---
M src/kudu/common/types.h
M src/kudu/common/wire_protocol-test.cc
2 files changed, 10 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/11911/3
-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 3
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11911/2/src/kudu/common/wire_protocol-test.cc
File src/kudu/common/wire_protocol-test.cc:

http://gerrit.cloudera.org:8080/#/c/11911/2/src/kudu/common/wire_protocol-test.cc@455
PS2, Line 455:   pb.set_read_default_value(string(16, 'a'));
> warning: length is bigger then string literal size [bugprone-string-constru
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 3
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 08 Nov 2018 22:23:50 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Grant Henke, Todd Lipcon, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11911

to look at the new patch set (#4).

Change subject: KUDU-2378: fix another aligned load crash
......................................................................

KUDU-2378: fix another aligned load crash

The TestKuduBackup.testRandomBackupAndRestore test from the kudu-backup
project would crash a TSAN master from time to time, with a stack trace
ending in Variant::Reset. After some debugging, this turned out to be a
holdover from KUDU-2378, where clang emitted a movaps (Aligned Load)
instruction on a value that wasn't aligned. The holdover was missed
because it was a static_cast (the original search covered
reinterpret_casts); I did a search for other bad casts but found none.

Why TSAN? Probably because it's the only build to use libc++, whose
std::string uses the Small String Optimization to embed the data into the
string itself, thus raising the possibility of the string's data being
unaligned. By comparison, libstdc++'s COW std::string uses a separate
allocation for string data, which is likely always 16-byte aligned.

The patch includes a unit test that would repro the crash 100% of the time
when built for TSAN.

Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
---
M src/kudu/common/types.h
M src/kudu/common/wire_protocol-test.cc
2 files changed, 11 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/11911/4
-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................

KUDU-2378: fix another aligned load crash

The TestKuduBackup.testRandomBackupAndRestore test from the kudu-backup
project would crash a TSAN master from time to time, with a stack trace
ending in Variant::Reset. After some debugging, this turned out to be a
holdover from KUDU-2378, where clang emitted a movaps (Aligned Load)
instruction on a value that wasn't aligned. The holdover was missed
because it was a static_cast (the original search covered
reinterpret_casts); I did a search for other bad casts but found none.

Why TSAN? Probably because it's the only build to use libc++, whose
std::string uses the Small String Optimization to embed the data into the
string itself, thus raising the possibility of the string's data being
unaligned. By comparison, libstdc++'s COW std::string uses a separate
allocation for string data, which is likely always 16-byte aligned.

The patch includes a unit test that would repro the crash 100% of the time
when built for TSAN.

Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Reviewed-on: http://gerrit.cloudera.org:8080/11911
Tested-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Todd Lipcon <to...@apache.org>
---
M src/kudu/common/types.h
M src/kudu/common/wire_protocol-test.cc
2 files changed, 11 insertions(+), 1 deletion(-)

Approvals:
  Adar Dembo: Verified
  Todd Lipcon: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 5
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Patch Set 2: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 08 Nov 2018 22:10:08 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2378: fix another aligned load crash

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11911 )

Change subject: KUDU-2378: fix another aligned load crash
......................................................................


Patch Set 4: Verified+1

Overriding Jenkins, DefaultSourceTest failed in TSAN again.


-- 
To view, visit http://gerrit.cloudera.org:8080/11911
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13e23a9d88db878071faeff7b9de06d1bdd98357
Gerrit-Change-Number: 11911
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 09 Nov 2018 01:03:26 +0000
Gerrit-HasComments: No